A Reinforcement Learning System with Chaotic Neural Networks-Based Adaptive Hierarchical Memory Structure for Autonomous Robots
|
|
- Earl Holt
- 5 years ago
- Views:
Transcription
1 Interntonl Conference on Control, Automton nd ystems 008 Oct. 4-7, 008 n COEX, eoul, Kore A Renforcement ernng ystem wth Chotc Neurl Networs-Bsed Adptve Herrchcl Memory tructure for Autonomous Robots Msno Obysh, Kenchro Nrt, Tsh Kuremoto nd Kunzu Kobysh Dvson of Computer cence & Desgn Engneerng, Ymguch Unversty, Ube, Jpn (Tel : ; E-ml:{ m.obys,wu,ob}@ymguch-u.c.p) Abstrct: Humn lerns ncdents by own ctons nd reflects them on the subsequent cton s own experences. These experences re memorzed n hs brn nd recollected f necessry. Ths reserch ncorportes such n ntellgent nformton processng mechnsm, nd pples t to n utonomous gent tht hs three mn functons: lernng, memorzton nd ssoctve recollecton. In the proposed system, n ctor-crtc type renforcement lernng method s used for lernng. Auto-ssoctve chotc neurl networ s lso used le mutul ssoctve memory system. Moreover, the memory prt hs n dptve herrchcl lyered structure of the memory module tht conssts of chotc neurl networs n consderton of the dustment to non-mdp (Mrov Decson Process) envronment. Fnlly, the effectveness of ths proposed method s verfed through the smulton ppled to the mze-serchng problem. Keywords: Renforcement lernng,chotc neurl networ, herrchcl memory structure, utonomous robot. INTRODUCTION Renforcement lernng (R..) s frmewor for n gent to lern the choce of n optml cton bsed on renforcement sgnl []. It hs been ppled to vrety of problem such s utonomous robot nvgton nd non-lner control nd so on. However, so fr, so mny systems wth R.. hve been mde up for only use of the one ts. Reserch for systems mng use of memorzng the results of lernng of mny tss nd pplyng them to other ts wthout lernng hs been lttle done. In ths study, we use the ssoctve chotc neurl networ (ACNN) proposed by Ahr et.l [] s storge mechnsm of results of R.. However, the storge cpcty of ACNN s smll, t s not sutble for worng lone. o, to resolve the problem, we mde up the herrchcl memory structure by mng use of ACNN: short-term memory for present lernng result, long-term memory for mny useful lernng results. Another chrcterstc of the proposed system s tht t s cpble of delng wth non-mdp problem n some degree, becuse of the chotc ssocton blty of ACNN. Fnlly t s verfed tht our proposed method s useful through the computer smulton for mze serchng problem.. PROPOED YTEM TRUCTURE The proposed system conssts of two prts: memory nd lernng. The memory conssts of short-term memory (.T.M.) nd long-term memory (.T.M.). Fg. shows ts overll structure. ernng sector : ctor-crtc system s dopted. It lerns the choce of cton to mxmze the totl predctve rewrds obtned over the future consderng the envronmentl nformton (s) nd rewrd (r) s result of cton ()..T.M. sector: t memorzes the lernng pth of the nformton (envronmentl nformton nd cton) obtned n ernng prt. Unnecessry nformton s forgotten nd useful nformton s stored..t.m. sector: t memorzes only the enough sophstcted nd useful experence n.t.m.. Autonomous gent ernng sector (ctor-crtc system) Memory hort-term memory (.T.M.) contents of TM Fg. Proposed system envronmentl nput s(t) nd rewrd r (t) cton (t) nformton bout pr of cton nd envronmentl one) contents of TM ong-term memory (.T.M.) s (t) Fg. The constructon of ctor-crtc system Envronmen 3. ACTOR-CRITIC REINFORCEMENT EARNIN YTEM The ctor-crtc renforcement lernng system s shown n Fg.. 69
2 3.. tructure nd lernng of crtc 3.. tructure Functon of the crtc s clculton of P(t) : the predcton vlue of sum of the dscounted rewrds tht wll be gotten over the future nd ts predcton error. These re shortly explned s follows; The sum of the dscounted rewrds tht wll be gotten over the future s defned s V () t. V n () t r( t + n), () =0γ n where γ ( 0 γ < ) s constnt clled dscount rte. Eq. () s rewrtten s () t = r() t + V ( t +) V γ. () Here the predcton vlue of V () t s defned s P () t. The predcton error rˆ () t s expressed s follows; () t = r() t + P( t + ) P() t rˆ γ. (3) The prmeters of the crtc re dusted to reduce ths predcton error rˆ () t. The predcton vlue P() t s clculted s follows; y P J () t y ( t) = ω (4) = 0 n ( x ( t) = exp = m ). (5) σ Here, ω : weght of the th output, y : th output of the mddle lyer of the crtc, x : th nput, m, σ : center, dsperson for th nput of th bss functon respectvely, J : the number of nodes n the mddle lyer of the crtc,. The constructon of the crtc s lso conssted of the RBFN s shown n Fg ernng ernng of crtc s done by usng commonly used Bc Propgton method whch mes predcton error rˆ () t goes to zero. Updtng rule of prmeters re s follows: ˆ c rt Δω = η c, ( =,, J ). (6) c ω 3.. tructure nd lernng of ctor 3.. tructure Fg.4 shows the constructon of the ctor. The ctor s bsclly conssted of Rdl Bss Functon Networ. The th bss functon of the mddle lyer node s s follows; Here y u m, σ n ( x = exp = J m ) σ, (7) () t y () t + n() t, ( =,, K) y : th = ω. (8) = output of the mddle lyer of the ctor, : center, dsperson for th nput of th bss functon respectvely, K: the number of the ctons, n : ddtve nose, u : representtve vlue of th cton, ω mddle lyer to : connecton weght from th th output. node of the 3.. Nose genertor Nose genertor let the output of the ctor hve the dversty by ddng the nose to t. It comes to relze the lernng of the trl nd error. Clculton of the nose n () t s s follows; n t = n = nose mn, exp( P t, (9) () t t ( ()) nose s unformly rndom number of [,] where t. As the P () t wll be bgger, the nose wll be smller. Ths leds to the stble lernng of the ctor ernng Prmeters of ctor, ω ( =,, J, =,, K), re dusted by usng output u of ctor nd nose n. u ˆ Δ ω = η nt rt ω, (0) η (> 0) s the lernng coeffcent. Eq. (0) mens tht ( nt δt ) s consdered s error, ω s dusted opposte to sgn of ( nt δt ). Fg.3 tructure of crtc Fg.4 tructure of ctor 70
3 3.3 Acton selecton The cton b t tme t s selected stochstclly usng bbs dstrbuton Eq.(). exp( ub () t T ) P( b x () t ) = K. () exp u t T = ( () ) Here, P( b x() t ) : selecton probblty of bth cton, b, T : postve constnt clled temperture constnt. 4. A HIERARCHICA MEMORY YTEM 4. Assoctve Chotc Neurl Networ (ACNN) CNN s constructed wth chotc neuron models tht hve refrctory nd contnuous output vlue. Its useful usge s s ssoctve memory networ nmed ACNN. Here re the dynmcs of ACNN. x ( t + ) = f ( y ( t + ) + z ( t + )), () y ( t + ) = y( t) α x ( t) +, (3) r z ( t + ) = z ( t) + ϖ x ( t), (4) f n = P p p ϖ = (x ) (x ), (5) P p= x (t) : output of the th neuron t tme t, y (t) : nternl stte respect to refrctory of the th neuron t tme t, z (t) : nternl stte respect to mutul operton of the th neuron t tme t., f ( ) : sgmod functon, ϖ : connecton weght from th neuron to th neuron, p x : th element of pth stored pttern. 4. Networ control Here, networ control s defned s control whch mes trnston of networ from chotc stte to non-chotc one nd vce vers. The networ control lgorthm of ACNN s shown n Fg.5. The stte of ACNN s clculted by Δ x(t), totl chnge of nternl stte x(t) temporlly, nd when Δx(t) s less thn threshold vlue θ, the chotc retrevl of ACNN s stopped by chngng vlues of prmeters r nto smll one. As result, networ converges to stored pttern ner the present networ stte. 4.3 Mutul ssoctve type ACNN 4.3. hort term memory(.t.m.) We me use of ACNN s mutul ssoctve memory system, nmely, uto-ssoctve mtrx s constructed wth envronmentl nputs s(t) nd ther correspondng ctons (t). When s (t) s set s ntl stte of ACNN, ACNN retreves (t) wth s (t) (refer to Fg.6). l s rndom vector to ween the correlton between s (t) nd (t). The memory mtrx W s descrbed s Eq.(6), here, λ s Fg.5 Networ control lgorthm Fg.6 Memory confgurton of ACNN Fg.7 Adptve herrchcl memory structure forgettng coeffcent, nd η s lernng coeffcent. λ s set to smll,becuse tht t ntl lernng stge s (t) s not correspondng to optml (t). W Input s new T [ s l ] [ s l ] old = λ W + η. (6) s Output l : ddtonl rndom memory unts for weenng correltons between s nd Actor - crtc system tored pttern [ s ( t) l( t) ( t) ] Input (t) s ACNN s mutul retrevl system Envronment.T.M. ACNN s mutul retrevl system Unt type memory structure 造.T.M. Unt type memory structure (0) Unt type memory structure () Output (t) 7
4 .T.M. s one unt conssts of plurl ACNNs, nd one ACNN memorzes nformton for one envronmentl nput pttern (refer to Fg.7). And.T.M hs pth nformton from strt to gol of only one mze serchng problem ong term memory(.t.m.) The.T.M. conssts of plurl unts. The.T.M. memorzes enough refned nformton n the.t.m. s one unt (refer to Fg.7). Nmely, when ctor-crtc lernng hs ccomplshed for certn mze problem, nformton n the.t.m. s updted s follows: In cse tht the present mze problem hs not been experenced, the stored mtrx W s set by Eq.(7) ; W = W. (7) In cse tht the present mze hs been experenced nd present lernng s ddtve lernng, the stored mtrx s updted by Eq.(8) ; new old W = λ W + η W. (8) λ s forgettng coeffcent, nd η s lernng coeffcent. λ s set to lrge vlue s sme s one of η so s not to forget prevous stored ptterns. 4.4 Adptve herrchcl memory structure Fg.7 shows whole confgurton of n dptve herrchcl memory structure. When n envronmentl stte s nputted to gent, t frst t s sent to the.t.m for confrmng f t s the stored nformton or not. If t s the stored nformton, the obtned cton correspondng to t s executed, otherwse, t s used to lern the ctor-crtc system. The pr of the enough refned nd trned envronmentl stte s nd cton n the.t.m. s sent to the.t.m. to be stored. If t s smlr to the stored pttern, nformton of the.t.m. s used to relern t the ctor-crtc system n the.t.m.. 5. COMPUTER IMUATION 5. multon condton Agent cn perceve whether there s sle or not t the forwrd, rght-forwrd, left-forwrd, rght, left s envronment s (refer to Fg.8). Agent cn move lttce to forwrd, bc, left, nd rght s cton (refer to Tble ). Therefore n ctor-crtc, stte s of envronment conssts of 5 nputs (= n). And nds of cton s 4(= K). The number of hdden nodes of R.B.F. s equl to 3(=J) n Fg. 3 nd 4. And the number of unts l s equl to n Fg.6. When gent gets the gol, gent s gven rewrd,.0. For the cse of collson wth wll, rewrd s -.0, nd for ech cton except collson s Other prmeters used n ths smulton re shown n Tble. 5. multon nd results 5.. In the cse of smple mze Fg.8 Perceptble re of gent : shded re Tble Acton code of gent Tble Prmeters of smulton ACTOR-CRITIC σ 0. ξ 0.7 η 0.3 γ 0.5 T Forgettng nd ernng coeffcents λ 0.89 η.00 λ.00 η.00 Chos control prmeters of ACNN Chos / Non-chos Chos / Non-chos α 0.0/.00 r 0.98/0.0 ε 0.05/0.05 f 0.0/00 T 0.3/ () Number of yer n the.t.m. (b) Fg.9 Expermentl mze nd results The num ber of stored ptterns yer yer 6073 yer 633 yer yer yer yer 6 73 yer yer yer totl 578 At frst, there s no dt n the.t.m., gent lerns the shortest pth of the mze of Fg.9() by usng the ctor-crtc system nd stores the result of lernng n the ACNN of the.t.m. correspondng to stte s n the form of Eq.(6). The fnl refned result for the me 7
5 s sent to be stored n the frst lyer (= unt(0) )of the.t.m. After lernng, gent restrted from the ntl poston nd got the nformton from ech lyer of the.t.m. nd got the gol le the rrow lne n Fg.9(). Fg.9(b) shows tht the number of stored ptterns concentrtes t yer 0, ths s becuse tht when gent goes ths mze gn, gent uses the nformton n unt(0) of the.t.m., but retrevl n ACNN fled on the wy, ll the nformton W of unt(0) moved to the.t.m. nd ddtonl ctor-crtc lernng ws done nd lernng results were wrtten ddtvely n the form of new Eq.(8) nd ll the nformton W ws sent to unt() of the.t.m. s new experenced nformton. Fg.9(b) shows the number of stored ptterns by Eq.(6) twce nd flure hppened t yer 0, so the number of stored ptterns concentrted t yer In the cse of lsng Agent moves eepng to te the posture such tht front of the gent s lwys upsde of the pper. In Fg.0, the optml pth t stte A s rght, however, ts pth t B s left. Agent perceves stte A nd B s sme stte, but ts optml ctons re dfferent, ths s clled lsng. In our cse, both ptterns re stored s dfferent ptterns. Our method solves ths problem by usng chos control of ACNN. Nmely, n the cse of sme ste, sme nput to ACNN, ACNN outputs ether left or rght s gent cton, consequently gent moves rght t A. () Non stored pth (b) tored pth (c) tored pth Fg. Expermentl mzes nd results () tored pth (b) tored pth A (c) tored pth 3 B Alsng hppens t these re Fg.0 Expermentl mze 5..3 In the cse of use of stored pth nformton At frst, there s no dt n the.t.m., gent lerns the shortest pth of the mze of Fg. (b) by usng the ctor-crtc system nd stores the result of lernng n the.t.m. n the form of Eq.(5) for ech cton. The fnl refned result s sent to be stored n the frst lyer ( = unt(0) ) of the.t.m. n shown Fg.7. econd, for the mze of Fg.(c), there s the pth nformton, gent tres to get the cton usng the ACNN of unt(0) n the.t.m., but fls becuse of no nformton correspondng to ths envronment. Agent lso lerns (d) Non stored pth Fg. Expermentl lrge scle mze 73
6 nd the fnl refned result s sent to the second lyer ( = unt() ) of the.t.m.. Fg.() shows the results tht gent s movng long the optml pth by mng use of experences (memory), tht s, (b) nd (c). The colored pth n Fg. () corresponds to those of (b) nd (c) In the cse of lrge scle mze After lernng of plurl smll sze mzes, Fg. () to (c), gent tred to get the gol for the mze n Fg. (d). Agents could not ssocte ts cton t the top of the rrow n Fg.(d). To get the gol n such lrge scle mze, mny experenced mzes re needed. 6. CONCUION We proposed renforcement lernng system wth chotc neurl networs-bsed dptve herrchcl memory structure for utonomous robots nd showed ts effectveness through gol serchng problem n plurl mzes. In our future wor, we would le to try to expnd ths method to be used n the cse of contnuous envronment. Acnowledgements A prt of ths study ws supported by JP-KAKENHI (No , No nd No ). REFERENCE [] R.. utton, A.. Brto:"Renforcement ern ng", The MIT Press,998 [] M. Adch, K. Ahr:"Assoctve Dynmcs n Chotc Neurl Networ", Neurl Networs,Vol. 0, No., pp.83-98,997 74
Dennis Bricker, 2001 Dept of Industrial Engineering The University of Iowa. MDP: Taxi page 1
Denns Brcker, 2001 Dept of Industrl Engneerng The Unversty of Iow MDP: Tx pge 1 A tx serves three djcent towns: A, B, nd C. Ech tme the tx dschrges pssenger, the drver must choose from three possble ctons:
More informationRank One Update And the Google Matrix by Al Bernstein Signal Science, LLC
Introducton Rnk One Updte And the Google Mtrx y Al Bernsten Sgnl Scence, LLC www.sgnlscence.net here re two dfferent wys to perform mtrx multplctons. he frst uses dot product formulton nd the second uses
More informationPartially Observable Systems. 1 Partially Observable Markov Decision Process (POMDP) Formalism
CS294-40 Lernng for Rootcs nd Control Lecture 10-9/30/2008 Lecturer: Peter Aeel Prtlly Oservle Systems Scre: Dvd Nchum Lecture outlne POMDP formlsm Pont-sed vlue terton Glol methods: polytree, enumerton,
More informationUNIVERSITY OF IOANNINA DEPARTMENT OF ECONOMICS. M.Sc. in Economics MICROECONOMIC THEORY I. Problem Set II
Mcroeconomc Theory I UNIVERSITY OF IOANNINA DEPARTMENT OF ECONOMICS MSc n Economcs MICROECONOMIC THEORY I Techng: A Lptns (Note: The number of ndctes exercse s dffculty level) ()True or flse? If V( y )
More informationRemember: Project Proposals are due April 11.
Bonformtcs ecture Notes Announcements Remember: Project Proposls re due Aprl. Clss 22 Aprl 4, 2002 A. Hdden Mrov Models. Defntons Emple - Consder the emple we tled bout n clss lst tme wth the cons. However,
More information4. Eccentric axial loading, cross-section core
. Eccentrc xl lodng, cross-secton core Introducton We re strtng to consder more generl cse when the xl force nd bxl bendng ct smultneousl n the cross-secton of the br. B vrtue of Snt-Vennt s prncple we
More informationThe Schur-Cohn Algorithm
Modelng, Estmton nd Otml Flterng n Sgnl Processng Mohmed Njm Coyrght 8, ISTE Ltd. Aendx F The Schur-Cohn Algorthm In ths endx, our m s to resent the Schur-Cohn lgorthm [] whch s often used s crteron for
More informationApplied Statistics Qualifier Examination
Appled Sttstcs Qulfer Exmnton Qul_june_8 Fll 8 Instructons: () The exmnton contns 4 Questons. You re to nswer 3 out of 4 of them. () You my use ny books nd clss notes tht you mght fnd helpful n solvng
More informationChapter Newton-Raphson Method of Solving a Nonlinear Equation
Chpter.4 Newton-Rphson Method of Solvng Nonlner Equton After redng ths chpter, you should be ble to:. derve the Newton-Rphson method formul,. develop the lgorthm of the Newton-Rphson method,. use the Newton-Rphson
More informationDCDM BUSINESS SCHOOL NUMERICAL METHODS (COS 233-8) Solutions to Assignment 3. x f(x)
DCDM BUSINESS SCHOOL NUMEICAL METHODS (COS -8) Solutons to Assgnment Queston Consder the followng dt: 5 f() 8 7 5 () Set up dfference tble through fourth dfferences. (b) Wht s the mnmum degree tht n nterpoltng
More informationIn this Chapter. Chap. 3 Markov chains and hidden Markov models. Probabilistic Models. Example: CpG Islands
In ths Chpter Chp. 3 Mrov chns nd hdden Mrov models Bontellgence bortory School of Computer Sc. & Eng. Seoul Ntonl Unversty Seoul 5-74, Kore The probblstc model for sequence nlyss HMM (hdden Mrov model)
More informationChapter Newton-Raphson Method of Solving a Nonlinear Equation
Chpter 0.04 Newton-Rphson Method o Solvng Nonlner Equton Ater redng ths chpter, you should be ble to:. derve the Newton-Rphson method ormul,. develop the lgorthm o the Newton-Rphson method,. use the Newton-Rphson
More informationElectrochemical Thermodynamics. Interfaces and Energy Conversion
CHE465/865, 2006-3, Lecture 6, 18 th Sep., 2006 Electrochemcl Thermodynmcs Interfces nd Energy Converson Where does the energy contrbuton F zϕ dn come from? Frst lw of thermodynmcs (conservton of energy):
More informationHaddow s Experiment:
schemtc drwng of Hddow's expermentl set-up movng pston non-contctng moton sensor bems of sprng steel poston vres to djust frequences blocks of sold steel shker Hddow s Experment: terr frm Theoretcl nd
More informationPrinciple Component Analysis
Prncple Component Anlyss Jng Go SUNY Bufflo Why Dmensonlty Reducton? We hve too mny dmensons o reson bout or obtn nsghts from o vsulze oo much nose n the dt Need to reduce them to smller set of fctors
More informationDynamic Power Management in a Mobile Multimedia System with Guaranteed Quality-of-Service
Dynmc Power Mngement n Moble Multmed System wth Gurnteed Qulty-of-Servce Qnru Qu, Qng Wu, nd Mssoud Pedrm Dept. of Electrcl Engneerng-Systems Unversty of Southern Clforn Los Angeles CA 90089 Outlne! Introducton
More informationQuiz: Experimental Physics Lab-I
Mxmum Mrks: 18 Totl tme llowed: 35 mn Quz: Expermentl Physcs Lb-I Nme: Roll no: Attempt ll questons. 1. In n experment, bll of mss 100 g s dropped from heght of 65 cm nto the snd contner, the mpct s clled
More informationGAUSS ELIMINATION. Consider the following system of algebraic linear equations
Numercl Anlyss for Engneers Germn Jordnn Unversty GAUSS ELIMINATION Consder the followng system of lgebrc lner equtons To solve the bove system usng clsscl methods, equton () s subtrcted from equton ()
More information523 P a g e. is measured through p. should be slower for lesser values of p and faster for greater values of p. If we set p*
R. Smpth Kumr, R. Kruthk, R. Rdhkrshnn / Interntonl Journl of Engneerng Reserch nd Applctons (IJERA) ISSN: 48-96 www.jer.com Vol., Issue 4, July-August 0, pp.5-58 Constructon Of Mxed Smplng Plns Indexed
More informationDefinition of Tracking
Trckng Defnton of Trckng Trckng: Generte some conclusons bout the moton of the scene, objects, or the cmer, gven sequence of mges. Knowng ths moton, predct where thngs re gong to project n the net mge,
More informationESCI 342 Atmospheric Dynamics I Lesson 1 Vectors and Vector Calculus
ESI 34 tmospherc Dnmcs I Lesson 1 Vectors nd Vector lculus Reference: Schum s Outlne Seres: Mthemtcl Hndbook of Formuls nd Tbles Suggested Redng: Mrtn Secton 1 OORDINTE SYSTEMS n orthonorml coordnte sstem
More informationLecture 4: Piecewise Cubic Interpolation
Lecture notes on Vrtonl nd Approxmte Methods n Appled Mthemtcs - A Perce UBC Lecture 4: Pecewse Cubc Interpolton Compled 6 August 7 In ths lecture we consder pecewse cubc nterpolton n whch cubc polynoml
More informationIntroduction to Numerical Integration Part II
Introducton to umercl Integrton Prt II CS 75/Mth 75 Brn T. Smth, UM, CS Dept. Sprng, 998 4/9/998 qud_ Intro to Gussn Qudrture s eore, the generl tretment chnges the ntegrton prolem to ndng the ntegrl w
More informationThe Number of Rows which Equal Certain Row
Interntonl Journl of Algebr, Vol 5, 011, no 30, 1481-1488 he Number of Rows whch Equl Certn Row Ahmd Hbl Deprtment of mthemtcs Fcult of Scences Dmscus unverst Dmscus, Sr hblhmd1@gmlcom Abstrct Let be X
More informationM/G/1/GD/ / System. ! Pollaczek-Khinchin (PK) Equation. ! Steady-state probabilities. ! Finding L, W q, W. ! π 0 = 1 ρ
M/G//GD/ / System! Pollcze-Khnchn (PK) Equton L q 2 2 λ σ s 2( + ρ ρ! Stedy-stte probbltes! π 0 ρ! Fndng L, q, ) 2 2 M/M/R/GD/K/K System! Drw the trnston dgrm! Derve the stedy-stte probbltes:! Fnd L,L
More informationA Family of Multivariate Abel Series Distributions. of Order k
Appled Mthemtcl Scences, Vol. 2, 2008, no. 45, 2239-2246 A Fmly of Multvrte Abel Seres Dstrbutons of Order k Rupk Gupt & Kshore K. Ds 2 Fculty of Scence & Technology, The Icf Unversty, Agrtl, Trpur, Ind
More informationTwo Activation Function Wavelet Network for the Identification of Functions with High Nonlinearity
Interntonl Journl of Engneerng & Computer Scence IJECS-IJENS Vol:1 No:04 81 Two Actvton Functon Wvelet Network for the Identfcton of Functons wth Hgh Nonlnerty Wsm Khld Abdulkder Abstrct-- The ntegrton
More information18.7 Artificial Neural Networks
310 18.7 Artfcl Neurl Networks Neuroscence hs hypotheszed tht mentl ctvty conssts prmrly of electrochemcl ctvty n networks of brn cells clled neurons Ths led McCulloch nd Ptts to devse ther mthemtcl model
More informationInvestigation phase in case of Bragg coupling
Journl of Th-Qr Unversty No.3 Vol.4 December/008 Investgton phse n cse of Brgg couplng Hder K. Mouhmd Deprtment of Physcs, College of Scence, Th-Qr, Unv. Mouhmd H. Abdullh Deprtment of Physcs, College
More informationLesson 2. Thermomechanical Measurements for Energy Systems (MENR) Measurements for Mechanical Systems and Production (MMER)
Lesson 2 Thermomechncl Mesurements for Energy Systems (MEN) Mesurements for Mechncl Systems nd Producton (MME) 1 A.Y. 2015-16 Zccr (no ) Del Prete A U The property A s clled: «mesurnd» the reference property
More informationSCALED GRADIENT DESCENT LEARNING RATE Reinforcement learning with light-seeking robot
SCALED GRADIET DESCET LEARIG RATE Renforcement lernng wth lght-seekng robot Kry Främlng Helsnk Unversty of Technology, P.O. Box 54, FI-5 HUT, Fnlnd. Eml: Kry.Frmlng@hut.f Keywords: Abstrct: Lner functon
More informationUsing Predictions in Online Optimization: Looking Forward with an Eye on the Past
Usng Predctons n Onlne Optmzton: Lookng Forwrd wth n Eye on the Pst Nngjun Chen Jont work wth Joshu Comden, Zhenhu Lu, Anshul Gndh, nd Adm Wermn 1 Predctons re crucl for decson mkng 2 Predctons re crucl
More information1 Online Learning and Regret Minimization
2.997 Decision-Mking in Lrge-Scle Systems My 10 MIT, Spring 2004 Hndout #29 Lecture Note 24 1 Online Lerning nd Regret Minimiztion In this lecture, we consider the problem of sequentil decision mking in
More informationFor the percentage of full time students at RCC the symbols would be:
Mth 17/171 Chpter 7- ypothesis Testing with One Smple This chpter is s simple s the previous one, except it is more interesting In this chpter we will test clims concerning the sme prmeters tht we worked
More informationEffects of polarization on the reflected wave
Lecture Notes. L Ros PPLIED OPTICS Effects of polrzton on the reflected wve Ref: The Feynmn Lectures on Physcs, Vol-I, Secton 33-6 Plne of ncdence Z Plne of nterfce Fg. 1 Y Y r 1 Glss r 1 Glss Fg. Reflecton
More informationName: SID: Discussion Session:
Nme: SID: Dscusson Sesson: hemcl Engneerng hermodynmcs -- Fll 008 uesdy, Octoer, 008 Merm I - 70 mnutes 00 onts otl losed Book nd Notes (5 ponts). onsder n del gs wth constnt het cpctes. Indcte whether
More information6.6 The Marquardt Algorithm
6.6 The Mqudt Algothm lmttons of the gdent nd Tylo expnson methods ecstng the Tylo expnson n tems of ch-sque devtves ecstng the gdent sech nto n tetve mtx fomlsm Mqudt's lgothm utomtclly combnes the gdent
More informationINTRODUCTION TO COMPLEX NUMBERS
INTRODUCTION TO COMPLEX NUMBERS The numers -4, -3, -, -1, 0, 1,, 3, 4 represent the negtve nd postve rel numers termed ntegers. As one frst lerns n mddle school they cn e thought of s unt dstnce spced
More informationDecomposition of Boolean Function Sets for Boolean Neural Networks
Decomposton of Boolen Functon Sets for Boolen Neurl Netorks Romn Kohut,, Bernd Stenbch Freberg Unverst of Mnng nd Technolog Insttute of Computer Scence Freberg (Schs), Germn Outlne Introducton Boolen Neuron
More information13.4 Work done by Constant Forces
13.4 Work done by Constnt Forces We will begin our discussion of the concept of work by nlyzing the motion of n object in one dimension cted on by constnt forces. Let s consider the following exmple: push
More informationProperties of Integrals, Indefinite Integrals. Goals: Definition of the Definite Integral Integral Calculations using Antiderivatives
Block #6: Properties of Integrls, Indefinite Integrls Gols: Definition of the Definite Integrl Integrl Clcultions using Antiderivtives Properties of Integrls The Indefinite Integrl 1 Riemnn Sums - 1 Riemnn
More informationSVMs for regression Multilayer neural networks
Lecture SVMs for regresson Muter neur netors Mos Husrecht mos@cs.ptt.edu 539 Sennott Squre Support vector mchne SVM SVM mmze the mrgn round the seprtng hperpne. he decson functon s fu specfed suset of
More informationUNSCENTED KALMAN FILTER POSITION ESTIMATION FOR AN AUTONOMOUS MOBILE ROBOT
Bulletn of the rnslvn Unversty of Brşov Vol. 3 (52) - 21 Seres I: Engneerng Scences UNSCENED KALMAN FILER OSIION ESIMAION FOR AN AUONOMOUS MOBILE ROBO C. SULIMAN 1 F. MOLDOVEANU 1 Abstrct: he Klmn flters
More informationState space systems analysis (continued) Stability. A. Definitions A system is said to be Asymptotically Stable (AS) when it satisfies
Stte spce systems nlysis (continued) Stbility A. Definitions A system is sid to be Asymptoticlly Stble (AS) when it stisfies ut () = 0, t > 0 lim xt () 0. t A system is AS if nd only if the impulse response
More informationReinforcement Learning
Reinforcement Lerning Tom Mitchell, Mchine Lerning, chpter 13 Outline Introduction Comprison with inductive lerning Mrkov Decision Processes: the model Optiml policy: The tsk Q Lerning: Q function Algorithm
More informationMany-Body Calculations of the Isotope Shift
Mny-Body Clcultons of the Isotope Shft W. R. Johnson Mrch 11, 1 1 Introducton Atomc energy levels re commonly evluted ssumng tht the nucler mss s nfnte. In ths report, we consder correctons to tomc levels
More informationSolution of Tutorial 5 Drive dynamics & control
ELEC463 Unversty of New South Wles School of Electrcl Engneerng & elecommunctons ELEC463 Electrc Drve Systems Queston Motor Soluton of utorl 5 Drve dynmcs & control 500 rev/mn = 5.3 rd/s 750 rted 4.3 Nm
More informationInternational Journal of Pure and Applied Sciences and Technology
Int. J. Pure Appl. Sc. Technol., () (), pp. 44-49 Interntonl Journl of Pure nd Appled Scences nd Technolog ISSN 9-67 Avlle onlne t www.jopst.n Reserch Pper Numercl Soluton for Non-Lner Fredholm Integrl
More informationState Estimation in TPN and PPN Guidance Laws by Using Unscented and Extended Kalman Filters
Stte Estmton n PN nd PPN Gudnce Lws by Usng Unscented nd Extended Klmn Flters S.H. oospour*, S. oospour**, mostf.sdollh*** Fculty of Electrcl nd Computer Engneerng, Unversty of brz, brz, Irn, *s.h.moospour@gml.com
More informationIdentification of Robot Arm s Joints Time-Varying Stiffness Under Loads
TELKOMNIKA, Vol.10, No.8, December 2012, pp. 2081~2087 e-issn: 2087-278X ccredted by DGHE (DIKTI), Decree No: 51/Dkt/Kep/2010 2081 Identfcton of Robot Arm s Jonts Tme-Vryng Stffness Under Lods Ru Xu 1,
More information6 Roots of Equations: Open Methods
HK Km Slghtly modfed 3//9, /8/6 Frstly wrtten t Mrch 5 6 Roots of Equtons: Open Methods Smple Fed-Pont Iterton Newton-Rphson Secnt Methods MATLAB Functon: fzero Polynomls Cse Study: Ppe Frcton Brcketng
More informationVariable time amplitude amplification and quantum algorithms for linear algebra. Andris Ambainis University of Latvia
Vrble tme mpltude mplfcton nd quntum lgorthms for lner lgebr Andrs Ambns Unversty of Ltv Tlk outlne. ew verson of mpltude mplfcton;. Quntum lgorthm for testng f A s sngulr; 3. Quntum lgorthm for solvng
More informationNew data structures to reduce data size and search time
New dt structures to reduce dt size nd serch time Tsuneo Kuwbr Deprtment of Informtion Sciences, Fculty of Science, Kngw University, Hirtsuk-shi, Jpn FIT2018 1D-1, No2, pp1-4 Copyright (c)2018 by The Institute
More informationChemical Reaction Engineering
Lecture 20 hemcl Recton Engneerng (RE) s the feld tht studes the rtes nd mechnsms of chemcl rectons nd the desgn of the rectors n whch they tke plce. Lst Lecture Energy Blnce Fundmentls F 0 E 0 F E Q W
More informationIntro to Nuclear and Particle Physics (5110)
Intro to Nucler nd Prticle Physics (5110) Feb, 009 The Nucler Mss Spectrum The Liquid Drop Model //009 1 E(MeV) n n(n-1)/ E/[ n(n-1)/] (MeV/pir) 1 C 16 O 0 Ne 4 Mg 7.7 14.44 19.17 8.48 4 5 6 6 10 15.4.41
More informationGoals: Determine how to calculate the area described by a function. Define the definite integral. Explore the relationship between the definite
Unit #8 : The Integrl Gols: Determine how to clculte the re described by function. Define the definite integrl. Eplore the reltionship between the definite integrl nd re. Eplore wys to estimte the definite
More informationDemand. Demand and Comparative Statics. Graphically. Marshallian Demand. ECON 370: Microeconomic Theory Summer 2004 Rice University Stanley Gilbert
Demnd Demnd nd Comrtve Sttcs ECON 370: Mcroeconomc Theory Summer 004 Rce Unversty Stnley Glbert Usng the tools we hve develoed u to ths ont, we cn now determne demnd for n ndvdul consumer We seek demnd
More informationMeasuring Electron Work Function in Metal
n experiment of the Electron topic Mesuring Electron Work Function in Metl Instructor: 梁生 Office: 7-318 Emil: shling@bjtu.edu.cn Purposes 1. To understnd the concept of electron work function in metl nd
More informationCENTROID (AĞIRLIK MERKEZİ )
CENTOD (ĞLK MEKEZİ ) centrod s geometrcl concept rsng from prllel forces. Tus, onl prllel forces possess centrod. Centrod s tougt of s te pont were te wole wegt of pscl od or sstem of prtcles s lumped.
More informationMinimal DFA. minimal DFA for L starting from any other
Miniml DFA Among the mny DFAs ccepting the sme regulr lnguge L, there is exctly one (up to renming of sttes) which hs the smllest possile numer of sttes. Moreover, it is possile to otin tht miniml DFA
More informationNUMERICAL MODELLING OF A CILIUM USING AN INTEGRAL EQUATION
NUEICAL ODELLING OF A CILIU USING AN INTEGAL EQUATION IHAI EBICAN, DANIEL IOAN Key words: Cl, Numercl nlyss, Electromgnetc feld, gnetton. The pper presents fst nd ccurte method to model the mgnetc behvour
More informationSupporting information How to concatenate the local attractors of subnetworks in the HPFP
n Effcen lgorh for Idenfyng Prry Phenoype rcors of Lrge-Scle Boolen Newor Sng-Mo Choo nd Kwng-Hyun Cho Depren of Mhecs Unversy of Ulsn Ulsn 446 Republc of Kore Depren of Bo nd Brn Engneerng Kore dvnced
More informationChemical Reaction Engineering
Lecture 20 hemcl Recton Engneerng (RE) s the feld tht studes the rtes nd mechnsms of chemcl rectons nd the desgn of the rectors n whch they tke plce. Lst Lecture Energy Blnce Fundmentls F E F E + Q! 0
More informationTwo Coefficients of the Dyson Product
Two Coeffcents of the Dyson Product rxv:07.460v mth.co 7 Nov 007 Lun Lv, Guoce Xn, nd Yue Zhou 3,,3 Center for Combntorcs, LPMC TJKLC Nnk Unversty, Tnjn 30007, P.R. Chn lvlun@cfc.nnk.edu.cn gn@nnk.edu.cn
More informationLecture 36. Finite Element Methods
CE 60: Numercl Methods Lecture 36 Fnte Element Methods Course Coordntor: Dr. Suresh A. Krth, Assocte Professor, Deprtment of Cvl Engneerng, IIT Guwht. In the lst clss, we dscussed on the ppromte methods
More information6. Chemical Potential and the Grand Partition Function
6. Chemcl Potentl nd the Grnd Prtton Functon ome Mth Fcts (see ppendx E for detls) If F() s n nlytc functon of stte vrles nd such tht df d pd then t follows: F F p lso snce F p F we cn conclude: p In other
More information4.4 Areas, Integrals and Antiderivatives
. res, integrls nd ntiderivtives 333. Ares, Integrls nd Antiderivtives This section explores properties of functions defined s res nd exmines some connections mong res, integrls nd ntiderivtives. In order
More informationInterpreting Integrals and the Fundamental Theorem
Interpreting Integrls nd the Fundmentl Theorem Tody, we go further in interpreting the mening of the definite integrl. Using Units to Aid Interprettion We lredy know tht if f(t) is the rte of chnge of
More informationReinforcement learning II
CS 1675 Introduction to Mchine Lerning Lecture 26 Reinforcement lerning II Milos Huskrecht milos@cs.pitt.edu 5329 Sennott Squre Reinforcement lerning Bsics: Input x Lerner Output Reinforcement r Critic
More informationINTERPOLATION(1) ELM1222 Numerical Analysis. ELM1222 Numerical Analysis Dr Muharrem Mercimek
ELM Numercl Anlss Dr Muhrrem Mercmek INTEPOLATION ELM Numercl Anlss Some of the contents re dopted from Lurene V. Fusett, Appled Numercl Anlss usng MATLAB. Prentce Hll Inc., 999 ELM Numercl Anlss Dr Muhrrem
More informationStudy of Trapezoidal Fuzzy Linear System of Equations S. M. Bargir 1, *, M. S. Bapat 2, J. D. Yadav 3 1
mercn Interntonl Journl of Reserch n cence Technology Engneerng & Mthemtcs vlble onlne t http://wwwsrnet IN (Prnt: 38-349 IN (Onlne: 38-3580 IN (CD-ROM: 38-369 IJRTEM s refereed ndexed peer-revewed multdscplnry
More informationADVANCEMENT OF THE CLOSELY COUPLED PROBES POTENTIAL DROP TECHNIQUE FOR NDE OF SURFACE CRACKS
ADVANCEMENT OF THE CLOSELY COUPLED PROBES POTENTIAL DROP TECHNIQUE FOR NDE OF SURFACE CRACKS F. Tkeo 1 nd M. Sk 1 Hchinohe Ntionl College of Technology, Hchinohe, Jpn; Tohoku University, Sendi, Jpn Abstrct:
More informationMultilayer Perceptron (MLP)
Multlayer Perceptron (MLP) Seungjn Cho Department of Computer Scence and Engneerng Pohang Unversty of Scence and Technology 77 Cheongam-ro, Nam-gu, Pohang 37673, Korea seungjn@postech.ac.kr 1 / 20 Outlne
More informationInfinite Geometric Series
Infinite Geometric Series Finite Geometric Series ( finite SUM) Let 0 < r < 1, nd let n be positive integer. Consider the finite sum It turns out there is simple lgebric expression tht is equivlent to
More informationAn Introduction to Support Vector Machines
An Introducton to Support Vector Mchnes Wht s good Decson Boundry? Consder two-clss, lnerly seprble clssfcton problem Clss How to fnd the lne (or hyperplne n n-dmensons, n>)? Any de? Clss Per Lug Mrtell
More informationMath 426: Probability Final Exam Practice
Mth 46: Probbility Finl Exm Prctice. Computtionl problems 4. Let T k (n) denote the number of prtitions of the set {,..., n} into k nonempty subsets, where k n. Argue tht T k (n) kt k (n ) + T k (n ) by
More informationTrade-offs in Optimization of GMDH-Type Neural Networks for Modelling of A Complex Process
Proceedngs of the 6th WSEAS Int. Conf. on Systems Theory & Scentfc Computton, Elound, Greece, August -3, 006 (pp48-5) Trde-offs n Optmzton of GDH-Type Neurl Networs for odellng of A Complex Process N.
More informationImproving Anytime Point-Based Value Iteration Using Principled Point Selections
In In Proceedngs of the Twenteth Interntonl Jont Conference on Artfcl Intellgence (IJCAI-7) Improvng Anytme Pont-Bsed Vlue Iterton Usng Prncpled Pont Selectons Mchel R. Jmes, Mchel E. Smples, nd Dmtr A.
More information2E Pattern Recognition Solutions to Introduction to Pattern Recognition, Chapter 2: Bayesian pattern classification
E395 - Pattern Recognton Solutons to Introducton to Pattern Recognton, Chapter : Bayesan pattern classfcaton Preface Ths document s a soluton manual for selected exercses from Introducton to Pattern Recognton
More informationComputing a complete histogram of an image in Log(n) steps and minimum expected memory requirements using hypercubes
Computng complete hstogrm of n mge n Log(n) steps nd mnmum expected memory requrements usng hypercubes TAREK M. SOBH School of Engneerng, Unversty of Brdgeport, Connectcut, USA. Abstrct Ths work frst revews
More informationMath 8 Winter 2015 Applications of Integration
Mth 8 Winter 205 Applictions of Integrtion Here re few importnt pplictions of integrtion. The pplictions you my see on n exm in this course include only the Net Chnge Theorem (which is relly just the Fundmentl
More informationJean Fernand Nguema LAMETA UFR Sciences Economiques Montpellier. Abstract
Stochstc domnnce on optml portfolo wth one rsk less nd two rsky ssets Jen Fernnd Nguem LAMETA UFR Scences Economques Montpeller Abstrct The pper provdes restrctons on the nvestor's utlty functon whch re
More informationOnline Learning Algorithms for Stochastic Water-Filling
Onlne Lernng Algorthms for Stochstc Wter-Fllng Y G nd Bhskr Krshnmchr Mng Hseh Deprtment of Electrcl Engneerng Unversty of Southern Clforn Los Angeles, CA 90089, USA Eml: {yg, bkrshn}@usc.edu Abstrct Wter-fllng
More information3/6/00. Reading Assignments. Outline. Hidden Markov Models: Explanation and Model Learning
3/6/ Hdden Mrkov Models: Explnton nd Model Lernng Brn C. Wllms 6.4/6.43 Sesson 2 9/3/ courtesy of JPL copyrght Brn Wllms, 2 Brn C. Wllms, copyrght 2 Redng Assgnments AIMA (Russell nd Norvg) Ch 5.-.3, 2.3
More informationResearch Article Special Issue
ournl of Fundmentl nd Appled Scences ISSN 1112-9867 Reserch Artcle Specl Issue Avlble onlne t http://www.fs.nfo A PERFORMANCE EVALUATION OF PRUNING EFFECTS ON HYBRID NEURAL NETWORK S. Y. Leow* 1, K. S.
More informationCALIBRATION OF SMALL AREA ESTIMATES IN BUSINESS SURVEYS
CALIBRATION OF SMALL AREA ESTIMATES IN BUSINESS SURVES Rodolphe Prm, Ntle Shlomo Southmpton Sttstcl Scences Reserch Insttute Unverst of Southmpton Unted Kngdom SAE, August 20 The BLUE-ETS Project s fnnced
More informationOperations with Polynomials
38 Chpter P Prerequisites P.4 Opertions with Polynomils Wht you should lern: How to identify the leding coefficients nd degrees of polynomils How to dd nd subtrct polynomils How to multiply polynomils
More informationWe partition C into n small arcs by forming a partition of [a, b] by picking s i as follows: a = s 0 < s 1 < < s n = b.
Mth 255 - Vector lculus II Notes 4.2 Pth nd Line Integrls We begin with discussion of pth integrls (the book clls them sclr line integrls). We will do this for function of two vribles, but these ides cn
More informationSparse and Overcomplete Representation: Finding Statistical Orders in Natural Images
Sprse nd Overcomplete Representton: Fndng Sttstcl Orders n Nturl Imges Amr Rez Sffr Azr Insttute for Theoretcl Computer Scence, Grz Unversty of Technology mr@g.tugrz.t Outlne Vsul Cortex. Sprse nd Overcomplete
More information19 Optimal behavior: Game theory
Intro. to Artificil Intelligence: Dle Schuurmns, Relu Ptrscu 1 19 Optiml behvior: Gme theory Adversril stte dynmics hve to ccount for worst cse Compute policy π : S A tht mximizes minimum rewrd Let S (,
More informationSolubilities and Thermodynamic Properties of SO 2 in Ionic
Solubltes nd Therodync Propertes of SO n Ionc Lquds Men Jn, Yucu Hou, b Weze Wu, *, Shuhng Ren nd Shdong Tn, L Xo, nd Zhgng Le Stte Key Lbortory of Checl Resource Engneerng, Beng Unversty of Checl Technology,
More informationTrigonometry. Trigonometry. Solutions. Curriculum Ready ACMMG: 223, 224, 245.
Trgonometry Trgonometry Solutons Currulum Redy CMMG:, 4, 4 www.mthlets.om Trgonometry Solutons Bss Pge questons. Identfy f the followng trngles re rght ngled or not. Trngles,, d, e re rght ngled ndted
More informationDesigning Information Devices and Systems I Discussion 8B
Lst Updted: 2018-10-17 19:40 1 EECS 16A Fll 2018 Designing Informtion Devices nd Systems I Discussion 8B 1. Why Bother With Thévenin Anywy? () Find Thévenin eqiuvlent for the circuit shown elow. 2kΩ 5V
More information8. INVERSE Z-TRANSFORM
8. INVERSE Z-TRANSFORM The proce by whch Z-trnform of tme ere, nmely X(), returned to the tme domn clled the nvere Z-trnform. The nvere Z-trnform defned by: Computer tudy Z X M-fle trn.m ued to fnd nvere
More informationReview of linear algebra. Nuno Vasconcelos UCSD
Revew of lner lgebr Nuno Vsconcelos UCSD Vector spces Defnton: vector spce s set H where ddton nd sclr multplcton re defned nd stsf: ) +( + ) (+ )+ 5) λ H 2) + + H 6) 3) H, + 7) λ(λ ) (λλ ) 4) H, - + 8)
More informationThis model contains two bonds per unit cell (one along the x-direction and the other along y). So we can rewrite the Hamiltonian as:
1 Problem set #1 1.1. A one-band model on a square lattce Fg. 1 Consder a square lattce wth only nearest-neghbor hoppngs (as shown n the fgure above): H t, j a a j (1.1) where,j stands for nearest neghbors
More informationScientific notation is a way of expressing really big numbers or really small numbers.
Scientific Nottion (Stndrd form) Scientific nottion is wy of expressing relly big numbers or relly smll numbers. It is most often used in scientific clcultions where the nlysis must be very precise. Scientific
More informationFrequency scaling simulation of Chua s circuit by automatic determination and control of step-size
Avlle onlne t www.scencedrect.com Appled Mthemtcs nd Computton 94 (7) 486 49 www.elsever.com/locte/mc Frequency sclng smulton of Chu s crcut y utomtc determnton nd control of step-sze E. Tlelo-Cuutle *,
More informationCHAPTER - 7. Firefly Algorithm based Strategic Bidding to Maximize Profit of IPPs in Competitive Electricity Market
CHAPTER - 7 Frefly Algorthm sed Strtegc Bddng to Mxmze Proft of IPPs n Compettve Electrcty Mrket 7. Introducton The renovton of electrc power systems plys mjor role on economc nd relle operton of power
More informationCHALMERS, GÖTEBORGS UNIVERSITET. SOLUTIONS to RE-EXAM for ARTIFICIAL NEURAL NETWORKS. COURSE CODES: FFR 135, FIM 720 GU, PhD
CHALMERS, GÖTEBORGS UNIVERSITET SOLUTIONS to RE-EXAM for ARTIFICIAL NEURAL NETWORKS COURSE CODES: FFR 35, FIM 72 GU, PhD Tme: Place: Teachers: Allowed materal: Not allowed: January 2, 28, at 8 3 2 3 SB
More information