Computational Verb Neural Networks

Transcription

1 INTERNATIONAL JOURNAL OF COMPUTATIONAL COGNITION VOL. 5, NO. 3, SEPTEMBER Computatoal Verb Neural Networks Tao Yag Abstract Whe ay attrbute value a covetoal eural etwork s verbfed, the result s a computatoal verb eural etworkvnn. We ca verbfy puts, outputs, bases, weghts ad may other attrbutes of covetoal eural etworks. I ths paper, we reported two types of VNNs. The frst oe cossts of computatoal verb puts ad umercal output. The secod oe cossts of computatoal verb puts ad outputs. The learg algorthms for both types of VNNs were provded. The exstece of solutos ad cotutes of sgle-layer feedforward type-i VNN were studed. Copyrght c 27 Yag s Scetfc Research Isttute, LLC. All rghts reserved. Idex Terms Computatoal verb eural etwork, VNN, computatoal verb, sgle-layer feedforward type-i VNN. I. INTRODUCTION ONE of the ma drawbacks of eural etworks s ts lack of hgh-level represetg power of the kowledge leart ad represeted ther weghts. Oe way to overcome ths problem s to mplemet the represetg power of atural laguages to the structures of eural etworks. The physcal lgustcs s the very framework to make atural laguage measurable ad therefore, the best tool to mplemet the represetg power of atural laguages to the structures of eural etworks. The author wll preset a eural etwork structure where computatoal verbs wll be workhorses to brg the hgh-level kowledge represetg power to the doma of measuremets. I [38], the author preseted the learg algorthms for sets of computatoal verb rules. These learg algorthms ca be readly trasformed to ft to the cofguratos of computatoal verb eural etworks. A computatoal verb eural etwork s the result of verbfyg a covetoal eural etwork. There are may ways to verbfy the attrbutes of a covetoal eural etwork. For example, oe ca verbfy the puts, outputs, weghts, ad bases of a covetoal eural etwork. The author wll study the followg two types of computatoal verb eural etworks. 1 Type-I computatoal verb eural etwork cossts of verbfed puts ad real outputs. All other attrbutes of are umercal. 2 Type-II computatoal verb eural etwork cossts of verbfed puts ad outputs. All other attrbutes of are umercal. Mauscrpt receved July 4, 27; revsed August 2, 27; May 27, 28. Tao Yag, Departmet of Electroc Egeerg, Xame Uversty, Xame 3615, P.R. Cha. Departmet of Cogtve Ecoomcs, Departmet of Electrcal Egeerg ad Computer Sceces, Yag s Scetfc Research Isttute, 133 East Uversty Blvd., #2882, Tucso, Arzoa , USA. Emal: taoyag@xmu.edu.c,taoyag@yagsky.com,taoyag@yagsky.us. Publsher Item Idetfer S /$2. Copyrght c 27 Yag s Scetfc Research Isttute, LLC. All rghts reserved. The ole verso posted o December 5, 27 at The orgazato of ths paper s as follows. I Secto II, the bref hstory of computatoal verb theory wll be gve. I Secto III, the structure of euro of type-i computatoal verb eural etworks ad ts learg algorthm wll be preseted. I Secto IV, the structure of euro of type-ii computatoal verb eural etworks ad ts learg algorthm wll be gve. I Secto V, the exstece of solutos ad the cotutes of sgle-layer feedforward type-i VNN wll be studed. I Secto VI, some cocludg remarks wll be preseted. II. A BRIEF HISTORY OF COMPUTATIONAL VERB THEORY As the frst paradgm shft for solvg egeerg problems by usg verbs, the computatoal verb theory ad physcal lgustcs have udergoe a rapd growth sce the brth of computatoal verb the Departmet of Electrcal Egeerg ad Computer Sceces, Uversty of Calfora at Berkeley 1997[8], [9]. The paradgm of mplemetg verbs maches were coed as computatoal verb theory[22]. The buldg blocks of computatoal theory are computatoal verbs[17], [12], [1], [18], [23]. The relato betwee verbs ad adverbs was mathematcally defed [11]. The logc operatos betwee verb statemets were studed [13]. The applcatos of verb logc to verb reasog were addressed [14] ad further studed [22]. A logc paradox was solved based o verb logc[19]. The mathematcal cocept of set was geeralzed to verb set [16]. Smlarly, for measurable attrbutes, the umber systems ca be geeralzed to verb umbers[2]. The applcatos of computatoal verbs to predctos were studed [15]. I [24] fuzzy dyamc systems were used to model a specal kd of computatoal verb that evolves a fuzzy space. The relato betwee computatoal verb theory ad tradtoal lgustcs was studed [22], [25]. The theoretcal bass of developg computatoal cogto from a ufed theory of fuzzy ad computatoal verb theores s the theory of the Ucogse that was studed [25], [3]. The ssues of smulatg cogto usg computatoal verbs were studed [26]. A way to mplemetg feelgs maches was proposed based o grouded computatoal verbs ad computatoal ous [32]. I [39] a ew defto of the smlarty betwee computatoal verbs was studed. The theory of computatoal verb has bee taught some uversty classrooms sce The latest actve applcatos of computatoal verb theory are lsted as follows. 1 Computatoal Verb Cotrollers. The applcatos of computatoal verbs to dfferet kds of cotrol prob- 1 Dr. G. Che, EE Itroducto to Fuzzy Iformatcs ad Itellget Systems, Departmet of Electroc Egeerg, Cty Uversty of Hog Kog. Dr. Mahr Sabra, EELE 636: Itellget Cotrol, Electrcal ad Computer Egeerg Departmet, The Islamc Uversty of Gaza.

2 58 INTERNATIONAL JOURNAL OF COMPUTATIONAL COGNITION VOL. 5, NO. 3, SEPTEMBER 27 lems were studed o dfferet occassos[21], [22]. For the advaced applcatos of computatoal verbs to cotrol problems, two papers reportg the latest advaces had bee publshed[28], [27]. The desg of computatoal verb cotroller was also preseted a textbook 26[1]. 2 Computatoal Verb Image Processg ad Image Uderstadg. The recet results of mage processg by usg computatoal verbs ca be foud [29]. The applcatos of computatoal verbs to mage uderstadg ca be foud [31]. 3 Stock Market Modelg ad Predcto based o computatoal verbs. The product of Cogtve Stock Charts[4] was based o the advaced modelg ad computg reported [33]. Applcatos of computatoal verbs was used to study the treds of stock markets kow as Russell recostructo patters [34]. Computatoal verb theory has bee successfully appled to may dustral ad commercal products. Some of these products are lsted as follows. 1 Vsual Card Couters. The YagSky-MAGIC card couter[6], developed by Yag s Scetfc Research Isttute ad Wux Xgcard Techology Co. Ltd., was the frst vsual card couter to use computatoal verb mage processg techology to acheve hgh accuracy of card ad paper board coutg based o cheap webcams. 2 CCTV Automatc Drver Qualfy Test System. The DrveQfy CCTV automatc drver qualfy test system[7] was the frst vehcle traectory recostructo ad stop tme measurg system usg computatoal verb mage processg techology. 3 Vsual Flame Detectg System. The FreEye vsual flame detectg system[2] was the frst CCTV or webcam based flame detectg system, that works uder color ad black & whte codtos, for survellace ad securty motorg system[36], [37]. 4 Smart Porographc Image ad Vdeo Detecto Systems. The PorSeer[5] porographc mage ad vdeo detecto systems are the frst cogtve feature based smart poro detecto ad removal software. 5 Webcam Barcode Scaer. The BarSeer[3] webcam barcode scaer took advatage of the computatoal verb mage processg to make the sca of barcode by usg cheap webcam possble. 6 Cogtve Stock Charts. By applyg computatoal verbs to the modelg of treds ad cogtve behavors of stock tradg actvtes, cogtve stock charts ca provde the traders wth the feelgs of stock markets by usg smple ad tutve dexes. III. TYPE-I COMPUTATIONAL VERB NEURAL NETWORKS A type-i computatoal verb eural etworkvnn 2 takes mplemetatos of computatoal verbs as puts, compares the put verbs wth some template verbs ad output real umbers. Therefore, a type-i VNN has verbs as puts ad 2 Istead of usg CVNN, we use VNN to stad for computatoal verb eural etwork to avod clutter. real umbers as outputs. I a type-i computatoal verb eural etwork, for put waveforms x t R, t [, T ], = 1,...,, there are template computatoal verbs V such that the computatoal verb smlarty betwee x t ad the evolvg fucto of V, E t, wll be weghted ad results the output y R of the euro as follow. y = f w Sx t, E t θ, t [, T ] 1 where S, s a computatoal verb smlarty[35], θ R s the bas of the euro ad the fucto f s the trasfer fucto for the euro. Gve m caocal computatoal verbs, {Ẽ} m, we ca represet a computatoal verb as E t = m α Ẽ t 2 where α R are costats ad ca be vewed as adverbs. Assume that we have K set of trag examples {u k1 t,..., u k t, d k } K, where u kt, = 1,...,, are put waveforms ad d k s the correspodg output. Our goal s to lear all computatoal verbs E t ad bas θ from the trag examples. For the kth trag sample, we costruct a output y k as, y k = f w Su k t, E t θ, 3 based o whch we costruct the followg error fucto. E = y k d k 2. 4 The learg rules are gve by w l + 1 = w l + γ w l w l, θl + 1 = θl + γ θ l θl, α l + 1 = α l + γ α l α l, l =,..., where γ w l R+, γ θ l R + ad γ α l R+, = 1,..., ; = 1,..., m, are learg rates for trag terato l, ad w E w y k d k 2 5 w 2 y k d k y k w 2 y k d k f w h Su kh t, E h t θ Su k t, E t, 6

3 YANG, COMPUTATIONAL VERB NEURAL NETWORKS 59 θ E θ y k d k 2 θ 2 y k d k y k θ = 2 y k d k f w h Su kh t, E h t θ, α E 2 = 2 y k d k 2 y k d k y k 7 y k d k f w h Su kh t, E h t θ w Su k t, E t 8 where Su kt,e t s gve by Eq. 1 where s t s a saturate fucto[35] gve by s t x = e x, ad ṡ e x tx = 1 + e x 2. 9 IV. TYPE-II COMPUTATIONAL VERB NEURAL NETWORKS A type-ii VNN has verbs as puts ad verbs as outputs. A euro a type-ii computatoal verb eural etwork s represeted as E y t = w Sx t, E te t E θ t, t [, T ] 11 where E y t s the evolvg fucto of the output computatoal verb, E θ t R s the verb bas of the euro. Gve m caocal computatoal verbs, {Ẽ} m, computatoal verbs E t, = 1,...,, are costructed as Eq. 2 ad E θ t s costructed as m E θ t = α θ Ẽt 12 where α θ R are costats ad ca be vewed as adverbs. Assume that we have K set of trag examples {u k1 t,..., u k t, d k t} K, where u kt, = 1,...,, are put waveforms ad d k t s the correspodg output verb. Our goal s to lear all computatoal verbs E t ad bas verb E θ t from the trag examples. For the kth trag sample, we costruct a output y k t as, y k t = w Su k t, E te t E θ t, 13 based o whch we costruct the followg error fucto. E = The learg rules are gve by w l + 1 = w l + γ w l w l, α θ l + 1 = α θ l + γ θ l α θ l, [y k t d k t] 2 dt. 14 α l + 1 = α l + γ α l α l, l =,..., 15 where γ w l R+, γ θ l R+ ad γ α l R+, = 1,..., ; = 1,..., m, are learg rates for trag terato l, ad w E w 2 2 α θ E α θ 2 = 2 [y k t d k t] 2 dt w [y k t d k t] y kt dt w [y k t d k t]su k t, E te t, α E 2 [y k t d k t] 2 dt α θ [y k t d k t] y kt α θ dt 16 [y k t d k t]ẽt, 17 [y k t d k t] 2 dt [y k t d k t] y kt dt 18

4 6 INTERNATIONAL JOURNAL OF COMPUTATIONAL COGNITION VOL. 5, NO. 3, SEPTEMBER 27 Su k t, E t = = 1 1 T s t u k t s t T 1 T 2 m s t u k t s t α Ẽ t dt m m α Ẽ t ṡ t α Ẽ t Ẽ tdt 2 m s t u k t s t α Ẽ t dt. 1 where y kt s gve by y k t Su k t, E t = w E t +w Su k t, E t E t = w E t Su kt, E t +w Su k t, E tẽt 19 where Su kt,e t s gve by Eq. 1. V. SINGLE-LAYER FEEDFORWARD TYPE-I VNN The structure of sgle-layer feedforward type-i VNN s show Fg. 1. We assume that there are put odes, m hdde odes ad 1 output ode. The weght w, = 1,..., ; = 1,..., m, coects betwee the th put ode ad the th hdde ode. The weght v coect the th hdde ode to the oly output ode. x 1 t x 2 t x t Fg. 1. Iput layer w f, θ f, θ f, θ Hdde layer v g, θ Output layer The structure of sgle-layer feedforward type-i VNN. The put-output relato of ths VNN s gve by m y = g v f w Sx t, E t θ θ 2 where each computatoal verb s costructed based o p caocal computatoal verbs p E t = α h Ẽ h t. 21 y Remark. If Eq. 2 we choose a computatoal verb smlarty ad a set of computatoal verbs such that Sx t, E t = x t ad oly cosder oe sample pot of the put waveforms, the VCNN 2 degeerates to the followg covetoal feedforward eural etwork wth oe hdde layer. m y = g v f w x θ θ. 22 The geeral cocluso s that all covetoal eural etworks ca be verbfed to computatoal verb eural etworks. Theorem 1. Assume that {u k t, d k }, k = 1,..., K, are K trag samples that are depedet ad have the same dstrbuto. u k t C[, T ] C[, T ]..., C[, T ] ad }{{} d k R. g s mootoc creasg. The for ay a gve approxmato error ε R +, the VNN 2 ca approxmate the put-output relato defed by the trag set f the umber of hdde odes, m, s bg eough. Proof. The kth trag sample s u k t, d k = u k 1,..., uk, d k. 23 It follows from Eq. 2 that the kth trag sample satsfes Eq. 24. The for the trag samples {u k t, d k }, k = 1,..., K ad a gve error ε Eq. 25 s satsfed. I Eq. 25 w, v, α h ad θ are adustable parameters to be determed. Therefore, Eq. 25 defes a rego the parameter space to clude the soluto to the put-output relato defed by the trag samples. The exstece of such a soluto s guarateed by prcples of olear programmg for each fxed θ whe we choose the total umber of parameters o less tha the umber of trag samples; amely, m+m+p K. Remark. It follows from the cocluso [38] that the set of olear equatos ca be solved umercally. The detals ca be foud from Sec. IV of [38]. Theorem 2. For VNN 2, assume that {x k t, y k }, k = 1, 2, are 2 put-output pars where x k t C[, T ] C[, T ]..., C[, T ] ad y k R, ad assume }{{} that f ad g are cotuous, the for ay ε >, there

5 YANG, COMPUTATIONAL VERB NEURAL NETWORKS 61 m d k = g v f w S m = g v f w S m v f w S m v f w S m v f w S u 1 u 2 u K t, t, t, u k u k t, E t t, θ θ p α h Ẽ h t θ θ. 24 p α h Ẽ h t θ = g 1 d 1 + θ, p α h Ẽ h t θ = g 1 d 2 + θ,.. p α h Ẽ h t θ = g 1 d K + θ. 25 exsts such a δ > that whe x 1 t x 2 t < δ, we have y 1 y 2 < ε. Proof. To avod clutter, let us use the followg otato. υ k w S x k t, E t θ, k = 1, Sce g s cotuous, for ay ε >, there exsts such a δ 1 > that whe m m v f υ 1 θ v f υ 2 θ m ] = v [f υ 1 f υ 2 < δ 1, 27 we have y 1 y 2 < ε. The ext step s to prove that for δ 1 >, there exsts such a δ > that whe x 1 t x 2 t < δ, Eq. 27 satsfes. Sce f s cotuous, for δ 1 >, there exst such a δ 2 > that whe υ 1 υ 2 < δ 2, we have fυ 1 fυ 2 < Therefore, f we choose a δ > satsfyg m v [f υ 1 f m v f f m m max = v υ 1 δ 1 m m max = v. 28 δ 1 υ 2 υ 2 ] m m max = v = δ VI. CONCLUDING REMARKS The results preseted ths paper revealed the brad ew opeg to a ew world of learg from dyamcal data or tme seres. The potetals of applyg VNN to facal world are tremedous f we vew the records of tra-day stock prces as dfferet computatoal verbs ad trasform the data-mg problem of tra-day stock prces to a learg problem of VNN. Needless to say, may smlar cases exst applcatos where dyamcal observatos are of regular tme wdows ad the kowledge dscovery from these data becomes mpossble f we oly vew each observatos solated, wthout cosderg the embedded atural tme-le them. It s the author s vso that wth te years, computatoal eural etwork aloe wll become a comprehesve dscple the famly of computatoal verb theory. REFERENCES [1] Guarog Che ad Trug Tat Pham. Itroducto to Fuzzy Systems. Chapma & Hall/CRC, November 25. ISBN: [2] Yag s Scetfc Research Isttute LLC. FreEye Vsual Flame Detectg Systems [3] Yag s Scetfc Research Isttute LLC. BarSeer Webcam Barcode Scaer [4] Yag s Scetfc Research Isttute LLC. Cogtve Stock Charts [5] Yag s Scetfc Research Isttute LLC. PorSeer Porographc Image ad Vdeo Detecto Systems [6] Yag s Scetfc Research Isttute LLC. ad Wux Xgcard Techology Ltd. YagSky-MAGIC Vsual Card Couters [7] Yag s Scetfc Research Isttute LLC. ad Chese Traffc Maagemet Research Isttute of the Mstry of Publc SecurtyTMRI-Cha. DrveQfy Automatc CCTV Drver Qualfy Testg Systems

6 62 INTERNATIONAL JOURNAL OF COMPUTATIONAL COGNITION VOL. 5, NO. 3, SEPTEMBER 27 [8] T. Yag. Verbal paradgms Part I: Modelg wth verbs. Techcal Report Memoradum No. UCB/ERL M97/64, Electrocs Research Laboratory, College of Egeerg, Uversty of Calfora, Berkeley, CA 9472, 9 Sept page [9] T. Yag. Verbal paradgms Part II: Computg wth verbs. Techcal Report Memoradum No. UCB/ERL M97/66, Electrocs Research Laboratory, College of Egeerg, Uversty of Calfora, Berkeley, CA 9472, 18 Sept page [1] T. Yag. Computatoal verb systems: Computg wth verbs ad applcatos. Iteratoal Joural of Geeral Systems, 281:1 36, [11] T. Yag. Computatoal verb systems: Adverbs ad adverbals as modfers of verbs. Iformato Sceces, :39 6, Dec [12] T. Yag. Computatoal verb systems: Modelg wth verbs ad applcatos. Iformato Sceces, : , Aug [13] T. Yag. Computatoal verb systems: Verb logc. Iteratoal Joural of Itellget Systems, 1411: , Nov [14] T. Yag. Computatoal verb systems: A ew paradgm for artfcal tellgece. Iformato Sceces A Iteratoal Joural, :13 123, 2. [15] T. Yag. Computatoal verb systems: Verb predctos ad ther applcatos. Iteratoal Joural of Itellget Systems, 1511: , Nov. 2. [16] T. Yag. Computatoal verb systems: Verb sets. Iteratoal Joural of Geeral Systems, 26: , 2. [17] T. Yag. Advaces Computatoal Verb Systems. Nova Scece Publshers, Ic., Hutgto, NY, May 21. ISBN [18] T. Yag. Computatoal verb systems: Computg wth perceptos of dyamcs. Iformato Sceces, : , Ju. 21. [19] T. Yag. Computatoal verb systems: The paradox of the lar. Iteratoal Joural of Itellget Systems, 169: , Sept. 21. [2] T. Yag. Computatoal verb systems: Verb umbers. Iteratoal Joural of Itellget Systems, 165: , May 21. [21] T. Yag. Impulsve Cotrol Theory, volume 272 of Lecture Notes Cotrol ad Iformato Sceces. Spger-Verlag, Berl, Aug. 21. ISBN X. [22] T. Yag. Computatoal Verb Theory: From Egeerg, Dyamc Systems to Physcal Lgustcs, volume 2 of YagSky.com Moographs Iformato Sceces. Yag s Scetfc Research Isttute, Tucso, AZ, Oct. 22. ISBN: [23] T. Yag. Computatoal verb systems: Verbs ad dyamc systems. Iteratoal Joural of Computatoal Cogto, 13:1 5, Sept. 23. [24] T. Yag. Fuzzy Dyamc Systems ad Computatoal Verbs Represeted by Fuzzy Mathematcs, volume 3 of YagSky.com Moographs Iformato Sceces. Yag s Scetfc Press, Tucso, AZ, Sept. 23. ISBN: [25] T. Yag. Physcal Lgustcs: Measurable Lgustcs ad Dualty Betwee Uverse ad Cogto, volume 5 of YagSky.com Moographs Iformato Sceces. Yag s Scetfc Press, Tucso, AZ, Dec. 24. [26] T. Yag. Smulatg huma cogto usg computatoal verb theory. Joural of Shagha UverstyNatural Sceces, 1s: , Oct. 24. [27] T. Yag. Archtectures of computatoal verb cotrollers: Towards a ew paradgm of tellget cotrol. Iteratoal Joural of Computatoal Cogto, 32:74 11, Jue 25 [avalable ole at http : // http : // [28] T. Yag. Applcatos of computatoal verbs to the desg of P-cotrollers. Iteratoal Joural of Computatoal Cogto, 32:52 6, Jue 25 [avalable ole at http : // http : // [29] T. Yag. Applcatos of computatoal verbs to dgtal mage processg. Iteratoal Joural of Computatoal Cogto, 33:31 4, September 25 [avalable ole at http : // http : // [3] T. Yag. Brdgg the Uverse ad the Cogto. Iteratoal Joural of Computatoal Cogto, 34:1 15, December 25 [avalable ole at http : // http : // [31] T. Yag. Applcatos of computatoal verbs to effectve ad realtme mage uderstadg. Iteratoal Joural of Computatoal Cogto, 41:49 67, March 26 [avalable ole at http : // http : // [32] T. Yag. Applcatos of computatoal verbs to feelg retreval from texts. Iteratoal Joural of Computatoal Cogto, 43:28 45, September 26 [avalable ole at http : // http : // [33] T. Yag. Applcatos of computatoal verbs to cogtve models of stock markets. Iteratoal Joural of Computatoal Cogto, 42:1 13, Jue 26 [avalable ole at http : // http : // [34] T. Yag. Applcatos of computatoal verbs to the study of the effects of Russell s aual dex recosttuto o stock markets. Iteratoal Joural of Computatoal Cogto, 43:1 8, September 26 [avalable ole at http : // http : // [35] T. Yag. Dstaces ad smlartes of saturated computatoal verbs. Iteratoal Joural of Computatoal Cogto, 44:62 77, December 26 [avalable ole at http : // http : // [36] T. Yag. Accurate vdeo flame-detectg system based o computatoal verb theory. AS Istaller, 42: , August 27. Chese. [37] T. Yag. Applcatos of computatoal verb theory to the desg of accurate vdeo flame-detectg systems. Iteratoal Joural of Computatoal Cogto, 53:25 42, September 27 [avalable ole at http : // http : // [38] T. Yag. Learg computatoal verb rules. Iteratoal Joural of Computatoal Cogto, 53:43 56, September 27 [avalable ole at http : // http : // [39] Ja Zhag ad Mru Fe. Determato of verb smlarty computatoal verb theory. Iteratoal Joural of Computatoal Cogto, 33:74 77, September 25 [avalable ole at http : // http : //