Learning of Situation Dependent Prediction toward Acquiring Physical Causality

Size: px

Start display at page:

Download "Learning of Situation Dependent Prediction toward Acquiring Physical Causality"

Vivian Short
5 years ago
Views:

1 Leaning of Siuaion Dependen Pedicion owad Acquiing Physical Causaliy Masaki Ogino Tesuya Fujia Sawa Fuke Minou Asada, JST ERATO Asada Synegisic Inelligence Pojec Yamadaoka 2-, Suia, Osaka , Japan Osaka Univesiy Gaduae School of Engineeing, Depamen of Adapive Machine Sysems, Yamadaoka 2-, Suia, Osaka , Japan Absac Physical causaliy is one of he mos impoan knowledge ha human babies lean fis afe bih hough ineacion wih he suounding envionmen. The popeies of objec movemen changes depending on he siuaion, and so he agen should change is pedicion. This pape poposes a leaning model which pedics he movemen of an aended objec depending on he envionmen aound he objec. The pedico is fomed by hee main layeed associaive modules: (a) an envionmen module, which ecognizes he aended objec and is suounding envionmen; (b) a pedico module, which anicipaes he movemen of he aended objec depending on he suounding envionmen; (c) an aenion module which implemens boom-up and op-down aenion pocesses. The poposed mehod is applied o he obo, and is pedicion faculy and adapabiliy ae examined in he simulaion and acual envionmen.. Inoducion All infans ae physiciss. Fom he day of he bih, hey begin o lean he fundamenal popeies of he wold sep by sep hough he ineacion wih he suounding envionmens. The one of he impoan indices fo hei pogess is objec pemanence ; even when an aended objec will be occluded fom he view by an obsacle, hey can undesand he objec will no be los fom he wold and emain behind obsacles. Alhough Piage fisly poposed ha infans can acquie objec pemanence afe 8 monh old (Piage, 954), ohe eseaches has shown ha infans can pass objec pemanence ask befoe one yea old (Baillageon e al., 985) (Baillageon and DeVos, 99). In developmenal cogniive oboics, some leaning models ae poposed o explain he esuls shown in hese expeimens wih moe esiced faciliies (Schlesinge, 23) (Love and Scasselai, 24). Alhough an acual mechanism ha enables an infan o show hese behavios even in such a ealy sage is sill unknown, how such highe conceps abou he wold as objec coninuiy and impossibiliy can be leaned auonomously is also an ineesing poblem in a obo aea (Fizpaick e al., 28). One of he fundamenal faculies o ealize he highe conceps abou he wold is o model he phenomena effecively fo appopiae pedicion. In his pape, we popose a leaning model ha enables a obo o lean he pedicion of he objec movemen depending on he siuaion. Fo his pupose, muliple Resiced Bolzmann Machines (RBMs) (Hinon e al., 26) ae adoped, which can be used fo boh unsupevised and supevised leanings. Moeove, wih his leaning model we ea he poblem on he elaionship beween aenion and leaning. Aenion is hough o consis of wo pocesses; boom-up and op-down aenion (Knudsen, 27). Wheeas boom-up aenion is modeled well by he ininsic feaues of he inpu image, op-down aenion is affeced by he expeience. So, wha o be aended is affeced by leaning, on he ohe hand wha is leaned is affeced by aenion. We se he aenion level based on he pedicion eo and how he aenion level affecs o he leaning. 2. Siuaion dependen pedico wih Resiced Bolzmann Machine 2. Oveview In ode o ealize a siuaion dependen pedico, he pedicion of he aended objec should be well meged wih ecogniion of he envionmen.

Pedicion Modules Pedicion Seleco siuaion Envionmen Module cuen sae nex sae Aenion Module " siuaion eain aenion elease! posiion and velociy!

as he model of inegaion of wo visual pahways (whee pahway and wha pahway) in he bain, whee appopiae self-oganizaion fo infomaion compession and inegaion should be

, 26) because his newok model possesses good feaues as a building uni o make a lage sysem. Fig.

The aenion module deemines he aenion aea in he envionmen.

The self oganized infomaion is associaed wih he infomaion of he movemen of he aended objec in he pedicion module.

siuaion. In his secion, ﬁs, he leaning algoihm of he esiced Bolzmann machine is explained.

2 Resiced Bolzmann Machine Resiced Bolzmann Machine (Hinon, 27) (Hinon e al., 26) is a neual newok consising of wo layes, inpu (visible) laye and hidden laye.

Each uni is acivaed by he following pobabiliies, P(hj ) P(vi ) + exp( + exp( i vi wij βhj ) () hj wij βvi ), (2) j whee βvi and βhj ae biases fo acivaion.

Fis, he acivaion lavel of he hidden laye, hj, ae calculaed by he fowad calculaion based on he inpu daa vi, he connecion weighs wij and biases βvi wih eq. ().

2 Pedicion Modules Pedicion Seleco siuaion Envionmen Module cuen sae nex sae Aenion Module " siuaion eain aenion elease! posiion and velociy! pedicion eo " Inpu Image Saliency Map Spaial ecogniion Objec ecogniion Objec Suounding Figue : Oveview of siuaion-dependen pedico This is also an ineesing poblem as he model of inegaion of wo visual pahways (whee pahway and wha pahway) in he bain, whee appopiae self-oganizaion fo infomaion compession and inegaion should be ealized. Fo ha pupose, we apply Resiced Bolzmann Machine (RBM) (Hinon e al., 26) because his newok model possesses good feaues as a building uni o make a lage sysem. Fig. shows a poposed sysem which consiss of 4 modules; aenion module, envionmen ecogniion module, pedico seleco and moion pedico. The aenion module deemines he aenion aea in he envionmen. Fom he aended aea, he geomeical infomaion of an aended objec and is suounding images ae exaced and self oganized by RBM in he envionmen ecogniion module. The self oganized infomaion is associaed wih he infomaion of he movemen of he aended objec in he pedicion module. Based on he associaion memoy feaue of RBM, he pedicion module can econsuc he nex posiion of he aended objec based on he cuen posiion and he cuen envionmenal siuaion. In his secion, ﬁs, he leaning algoihm of he esiced Bolzmann machine is explained. Second, i is explained how RBMs ae used in he envionmen ecogniion module and pedicion module. Then, he aenion module is explained 2.2 Resiced Bolzmann Machine Resiced Bolzmann Machine (Hinon, 27) (Hinon e al., 26) is a neual newok consising of wo layes, inpu (visible) laye and hidden laye. Thee ae no connecions among unis wihin each laye. Each uni in he visible laye, vi, has a symmeical connecion weigh, wij, o each uni in he hidden uni, hj. Each uni is acivaed by he following pobabiliies, P(hj ) P(vi ) + exp( + exp( i vi wij βhj ) () hj wij βvi ), (2) j whee βvi and βhj ae biases fo acivaion. The leaning of RBM is pocessed by he calculaion pocess called econsucion. Fis, he acivaion lavel of he hidden laye, hj, ae calculaed by he fowad calculaion based on he inpu daa vi, he connecion weighs wij and biases βvi wih eq. (). Then he acivaion level of he visible laye, vi, is calculaed again wih he acivaion level of he hidden laye, hj wih eq. (2). In he following, his e-calculaed acivaion level of he visible laye is called econsucion daa. This calculaion pocess can be poceeded epeaedly. The supescip of he uni vi and hj menions he numbe of he epeaed calculaion beween layes. When he pobabilisic disibuion of he inpu daa and he econsuced daa afe epeas of econsucion ae p (v) and p (v w), especively, he pupose of he leaning is o adjus he connecion weighs, wij, o minimize he diﬀeence of he disibuion beween p (v) and p (v w). The disance beween wo disibuions can be measued by he coss enopy eo, which is deﬁned by he following equaion, L hlog(p(v w))ip (x) N i p (vi ) log(p(vi w)). (3) (4)

3 The oal enegy of he RBM newok wih he acivaion level, (v, h) in he boh layes, can be defined by he following equaion, E(v, h w) i,j v i h j w ij. (5) In he same way, he leaning ule fo biases can be deived as β hj ɛ(p(h j ) P(h j )) (4) β vi ɛ(p(v i ) P(v i )) (5) The pobabiliy of he ealizaion of he sae (v, h) is popoional o he oal enegy, p(v, h w) e E(v,h w). (6) Thus, when he funcion of he igh side of he equaion is descibed as f like, f(v, h w) e E(v,h w) (7) f(v w) h e E(v,h w), (8) hen he pobabiliies of he ealizaion of he sae (v, h) and bfv given he weighs bfw can be wien wih f as p(v, h) p(v) f(v, h w) v,h f(v, h w) (9) h f(v, h w) f(v w) f(v, h w) v f(v w).() v,h Applying he elaion log p(v w) log f(v w) v log f(v w), he deivaion of he coss enopy eo, 4, can be ansfomed as follows, L w w log f(v w) v w log f(v w) v f(v w) Z p(v w) w w log f(v w) ˆp(x) w log f(v w) p(v w) w log f(v w) p w log f(v w) p whee p is he inpu daa ( h econsucion daa) and p is he -h econsucion daa. Fo acual calculaion, insead of p, s econsucion daa, p, is used fo minimizaion. Then, he deivaion can be simplified as log f(v w ij ) p log f(v w ij ) p w ij w ij v i h j w ij p v i h j w ij p () w ij w ij i,j i,j log f(v w) ˆp(x) w 2.3 Envionmen Module v i h j p v i h j p (2) Thus, he updae leaning ule fo minimizing he coss enopy eo can be deived as In he acual leaning, he inpu daa ae divided ino seveal goups and he paamees ae updaed goup by goup o avoid he ove leaning. Moeove, we added he addiional of leaning ule o limi he acivaion ae of each uni. This spaseness consain seems o be impoan o descibe he inpu daa wih moe compac paens of acivaions in hidden layes (Lee e al., 28). The convegence of he leaning is evaluaed by he oal eo beween inpu daa and he econsucion daa, e v i P(v i ). (6) Afe leaning, he econsucion pocess can be used fo econsucing complee daa se fom he incomplee daa. This feaue can be used fo associaion of he given muliple daa ses. Moeove, when he numbe of he unis in he hidden laye is less han ha in he visible laye, he exacion of he impoan feaues of he inpu daa can be expeced. Hinon sesses ha his chaaceisic of RBM favoable fo avoiding local minima in leaning of deep layeed newok. Thus, RBM has boh feaues of supevised and unsupevised self-oganizaion leaning popeies. log An objec will change he movemen depending on f(v w) ˆp(x) he envionmen whee he objec is pu. The popeies of he movemen will be affeced by he shape of he objec. Fo example, we expec a ball shape will be expeced o move easily bu no fo a squae objec. And he same objec will change is movemen depending on he pahway he objec is pu on. The envionmen module caegoies visual infomaion of an aended objec and is suoundings. To exac he geomeical infomaion fom he images of an aended objec and is suoundings, he esuls of he gabo files of hese images ae inpu o RBM. The esul images of he gabo files of ψ [, 45, 9, 35 ] ae segmened ino 5 5 unis. In each uni, he pixel values ae summed and nomalized o he aended aea. The acivaion level of he j-h uni, I j, is deemined by he nomalized summed value a j and some heshold, I j (a j > h) (a j h). (7) w ij ɛ(v i P(h j ) P(v i )P(h j )). (3) The inpu o he envionmen module RBM, v <env>, is he combinaion of he vecos of he gabo file

esuls fo he objec image, I <obj>, and he vecos of he gabo file esuls fo he suounding image, I <aound>, v <env> (I <obj>, I <aound> ). (8) Fig. 2 shows he flow cha of he pocessing.

Thus, hese infomaion is used in he pedicion module fo pedicion of he movemen of he aended objec.

in g im a g e Figue 2: The envionmen module 2.4 Pedicion Module Fig. 3 shows he schema of he pedicion module.

The posiion infomaion consiss of he posiion and he velociy of an aended objec, S() (x, x,..., x n, y, y,..., y m, dx, dx,..., dx 2n, dy.dy,..., dy 2m ).

(22) else In ode o ealize he pedicion in he vaious ime scales and spaial fames, seveal RBM wih vaious kinds of ime scale and spaial segmenaion sizes ae pepaed.

The eliabiliy of i-h pedico, c i, is calculaed based on he hidden laye of he envionmen ecogniion module, h <env>, as c i j w s ij h <env> j.

4 esuls fo he objec image, I <obj>, and he vecos of he gabo file esuls fo he suounding image, I <aound>, v <env> (I <obj>, I <aound> ). (8) Fig. 2 shows he flow cha of he pocessing. Afe leaning, he acivaion paen in he hidden laye, h <env>, is expeced o descibe self oganized infomaion of he inpu images. Thus, hese infomaion is used in he pedicion module fo pedicion of he movemen of he aended objec. v < e n v> ( I < o bj>, I < a o u n d > ) I < o b j > I < a o u n d > o b j e c im a g e b in a iz e d g a b o fi l e E n v i o n m e n M o d u le < e n v > b in a iz e d g a b o fi l e s u o u n d in g im a g e Figue 2: The envionmen module 2.4 Pedicion Module Fig. 3 shows he schema of he pedicion module. The RBM in pedicion module associaes he cuen movemen infomaion S(), he pevious movemen infomaion S( ), and he siuaion infomaion h <env>. The posiion infomaion consiss of he posiion and he velociy of an aended objec, S() (x, x,..., x n, y, y,..., y m, dx, dx,..., dx 2n, dy.dy,..., dy 2m ). When he image size is W H and i is divided ino n m, and he coodinaes of he aended objec ae (x, y), hen he posiion nodes ae deemined by he following equaions, ( x W/n x i i < x W/n + ) else (9) and ( y H/m y j j < y H/m + ). (2) else When he shif of he aended objec beween obseved seps is (dx, dy), he velociy nodes ae deemined by ( dx W/n dx i + n i < else dx W/n + n) (2) and ( dy H/m dy j + m j < dy H/m + m). (22) else In ode o ealize he pedicion in he vaious ime scales and spaial fames, seveal RBM wih vaious kinds of ime scale and spaial segmenaion sizes ae pepaed. Among hem, he appopiae pedico is seleced based on he eliabiliy of he pedicos. The eliabiliy of i-h pedico, c i, is calculaed based on he hidden laye of he envionmen ecogniion module, h <env>, as c i j w s ij h <env> j. (23) The connecion weighs, wij s, is leaned based on he following Hebbian leaning, w s ij ɛ(e i h <env> j ) (24) whee i is he pedicion eo of he movemen in i-h pedicion module, ɛ is he leaning ae. The acivaion level of each RBM, a <RBM> i, is calculaed based on he eliabiliy c i, a <RBM> i + exp( i c i) (25) and he RBM ha has he maximum value is seleced as he pedico unde he cuen siuaion. 2.5 Aenion Module We hypohesized ha aenion consiss of hee pocesses; cach, eain and elease. Fis, in he cach pocess, he aended poin is seleced based on he saliency (Ii e al., 23). Fo ha pupose, he saliency map is calculaed wih egad o vaious image feaues such as inensiy, colo, moion, ec. Once he aended poin is decided, he aended objec aea is evaluaed as he se of pixels ha have he same colo of aended poin. Then, he aended objec aea is segmened and used fo he emplae fo paen maching. The objec ae is nomalized and binaized as he inpu fo he aenion module, I <obj> (Fig. 2). The suounding image of he aended objec whose size is he half of he camea image is nomalized and binaized as he inpu fo he

Module posiion and displacemen To validae he pedicion faculy, he poposed sysem is

Alhough his obo has wo IEEE 394 cameas and 2 degees of feedom (pan and il) o move

The camea image is capued wih 33 [fames/sec].

The aenion poin is eained fo he leaning unil he igge fo eleasing aended poin is

Fo eﬀecive leaning, i is supposed ha he aended poins whose movemen can be pediced

The poins whose movemen ae andom should also be eleased ealie because such poins

On he ohe hand, he poins whose movemen can be paly pediced should be eained long fo

Fo ha pupose, we compaed wo kinds of funcions ha decide he pobabiliies o elease he

5 + e + e neo (27) e whee neo is he ae of he numbe of he pedicion modules ha fails

The aenion is eleased when he above aenion level becomes less han some heshold.

siuaions ae pepaed; a ball on he holizonal ail, a ball on he veical ail and a ball

5 An h <an> 3. A2 A Expeimens 3. Leaning Pedicion wihou aenion Pedicion Module v < an> (S(+),S(),h< env>) Envionmen Module posiion and displacemen To validae he pedicion faculy, he poposed sysem is applied o he eal obo. Fig. 5 shows he obo FK used in he expeimen. Alhough his obo has wo IEEE 394 cameas and 2 degees of feedom (pan and il) o move he camea, only he igh camea is used wih eye posiion ﬁxed. The camea image is capued wih 33 [fames/sec]. Figue 3: The pedicion module aenion module, I <aound>. The aenion poin is eained fo he leaning unil he igge fo eleasing aended poin is given. Fo eﬀecive leaning, i is supposed ha he aended poins whose movemen can be pediced compleely should be eleased. The poins whose movemen ae andom should also be eleased ealie because such poins may well be noise. On he ohe hand, he poins whose movemen can be paly pediced should be eained long fo leaning moe. Fo ha pupose, we compaed wo kinds of funcions ha decide he pobabiliies o elease he aenion poins. aenion aenion2 eneo e neo + e + (26) 2e.5 + e + e neo (27) e whee neo is he ae of he numbe of he pedicion modules ha fails o pedic. The gaphs ae shown in Figs. 4. The aenion is eleased when he above aenion level becomes less han some heshold. Figue 5: obo FK To validae he pedicion faculy in vaious siuaions, hee kinds of siuaions ae pepaed; a ball on he holizonal ail, a ball on he veical ail and a ball in he pendulum 6. In each siuaion, 4 i- (a) siuaion (b) siuaion 2 (c) siuaion 3. Figue 6: Siuaions of expeimens. n n i o i n e A A n o e. o o e. e (a) Aenion funcion (b) Aenion funcion2 Figue 4: Aenion funcion als ae ecoded, each of which has abou 9 seps. Fo he pedicion module, 6 RBMs ae pepaed (2 kinds of segmenaions (4 4, ) and 3 kinds of ime seps (5,, 2 seps)). The numbes of he visible and hidden unis of he envionmen module ae 2 and 5, especively. The numbes of he visible and hidden unis of he pedicion modules ae 368 and 92 fo he segmen size 4 4, 28 and 32 fo he segmen size. The aended aea o be

.3.2 (Δ, 4x4) (Δ2, 4x4) (Δ5, x) (Δ, x) (Δ5, 4x4). Pedicion Eo (a) Siuaion (Δ2, x) (b) Siuaion 2. Figue 9: The aining daa fo supplemenal leaning 5 5 Leaning Seps Figue 7: Pedicion Eo wihou aenion.

The examples of he pedicion afe leaning is menioned in Fig. 8. These ae he pedicions of RBMs ha have segmens and 5,, 2 pedicion ime seps (2 sep pedicion is shown only in evey 2 sep). given o he newok.

The daa of second siuaion is added o he aining daa of he siuaion newok afe he 25 seps of leaning in he ﬁs siuaion. Fig.

Howeve, in he following seps, he eo ae deceases o aound.3 indicaing ha he newok successfully epesen boh he new and old siuaion. Fig.

6 .3.2 (Δ, 4x4) (Δ2, 4x4) (Δ5, x) (Δ, x) (Δ5, 4x4). Pedicion Eo (a) Siuaion (Δ2, x) (b) Siuaion 2. Figue 9: The aining daa fo supplemenal leaning 5 5 Leaning Seps Figue 7: Pedicion Eo wihou aenion. aended is given as he image emplae (oage ball) by he designe in his expeimen. Fig. 7 shows he leaning eo of all RBMs. The leaning of each RBM conveges wihin leaning seps. The examples of he pedicion afe leaning is menioned in Fig. 8. These ae he pedicions of RBMs ha have segmens and 5,, 2 pedicion ime seps (2 sep pedicion is shown only in evey 2 sep). given o he newok. Fo each siuaion, 3 ials (each ial consiss of 68 seps) ae ecoded fo aining daa. The ohe condiions ae he same as he pevious subsecion. The daa of second siuaion is added o he aining daa of he siuaion newok afe he 25 seps of leaning in he ﬁs siuaion. Fig. shows he ime couses of he aveaged eo ae pe one node hough he leaning pocess (only 2 of 6 pedicos ae shown). Aound he 25-h leaning seps, he eo ae ises when new daa is added o he aining daa. Howeve, in he following seps, he eo ae deceases o aound.3 indicaing ha he newok successfully epesen boh he new and old siuaion. Fig. 5 2 (Δ5, 4x4) 5 2 (d) Sep (e) Sep (f) Sep (c) Sep (b) Sep (Δ, 4x4). (a) Sep Pedicion Eo Rae Leaning Seps Figue : The eo ae pe one node in he supplemenal leaning (g) Sep 6 (h) Sep 7 (i) Sep 8 Figue 8: Examples of pedicion of he movemen afe leaning 3.. Supplemenal Leaning To validae he faculy of he pedico in addiional leaning, afe leaning in one siuaion (Fig. 9(a)), addiional daa in anohe siuaion (Fig. 9(b)) is shows he pediced posiion of he aended objec befoe and afe he supplemenal leaning. (a) Befoe he supplemenal leaning, he obo pedics he aended ball will go hough he wall because he does no expeienced such kind of siuaion (squaes ae he pediced posiions in he nex 5, and 2 seps. The big and small squaes ae he pedicion in and 4 4 segmens.). (b) Afe he supplemenal leaning, he obo can make appopiae pedicions depending on he siuaions. Befoe he supplemenal leaning, he obo pedics ha

Pedicion Eo Pedicion Eo he ball will move hough he wall in igh diecion as befoe (b). Afe he supplemenal leaning, he obo can pedic ha he ball will sop a he wall (c).

Wih he funcion, he obo pedics he fis pa of he movemen successfully and loses is aenion easily because he pedicion is successfully done. This makes he leaning unsable.

(a) Pedicion befoe leaning (b) Pedicion afe leaning (siuaon ).5.4 x 5 2.3.2 4x4 5 2 (c) Pedicion afe leaning (siuaion 2) Figue : Pedicion of he ball posiion befoe and afe he supplemenal leaning...4.3.2 Time Sep 5 2 Segmen Size 4x4 x 5 5 2 25 Leaning Seps Time Sep 5 2 Segmen Size 4x4 x 3.

7 Pedicion Eo Pedicion Eo he ball will move hough he wall in igh diecion as befoe (b). Afe he supplemenal leaning, he obo can pedic ha he ball will sop a he wall (c). x 5 2 4x4 5 2 x 5 2 4x4 5 2 line indicaes he heshold ha he obo changes is aenion (he aows indicae he iming when he obo changes is aenion). Wih he funcion, he obo pedics he fis pa of he movemen successfully and loses is aenion easily because he pedicion is successfully done. This makes he leaning unsable. On he ohe hand, wih he funcion 2, he obo can keep is aenion once he appopiae aenion poin (he objec) is found. And he sable leaning daa can be obained. (a) Pedicion befoe leaning (b) Pedicion afe leaning (siuaon ).5.4 x x4 5 2 (c) Pedicion afe leaning (siuaion 2) Figue : Pedicion of he ball posiion befoe and afe he supplemenal leaning Time Sep 5 2 Segmen Size 4x4 x Leaning Seps Time Sep 5 2 Segmen Size 4x4 x 3.2 Leaning Pedicion wih Aenion. In he expeimens of pevious subsecion, he objec o be aended is given by he designe in advance. In ode fo a obo o lean he physical causaliy auonomously, i is impoan o implemen an aenion conol sysem appopiaely. Fo his pupose, we applied he aenion module o he same siuaions as he expeimens explained in he pevious subsecion. Each siuaion consiss of 3 ials ha have 9 seps. Fo he pedicion module, 6 RBMs ae pepaed (2 kinds of segmenaions (4 4, ) and 3 kinds of ime seps (5,, 2 seps)) fo he pedicion modules. Figs. 2 shows he ime couse of he eo ae in he leaning pocedues wih he aenion funcion (Fig. 2 (above)) and he aenion funcion 2 (Fig. 2 (below)). Wheeas he leaning is no sable wih he aenion funcion, he leaning wih he aenion funcion 2 conveges o some sable sae. This is because wih he aenion funcion he obo easily change is aenion o anohe poin (ofen shiny noise poin in he envionmen ohe han he objec) when he fis pa of he movemen can be leaned. Figs. 3 show he iming when he obo changes is aenion in he middle of he leaning pocedue fo siuaion 3 wih he funcion (above) and wih he funcion 2 (below). In hese gaphs, he gay line indicaes he ae of he pedicion failue modules, he black line indicaes he aenion level (calculaed by eq. (26) and eq. (27)), and he dashed Leaning Seps Figue 2: Pedicion eo based on he aenion module wih he funcion (above) and he funcion 2 (below) Time [sep] Time [sep] eo/oal aenion eo/oal aenion Figue 3: Aenion level and aenion changes based on he aenion module wih he funcion (above) and he funcion 2 (below)

8 4. Discussion In his pape, we poposed a layeed associaive newok ha can pedic he movemens of he obseved objec depending on he suounding siuaion. The highe absac concep such as objec pemanence can be acquied hough he leaning of many concee phenomena in he eal wold. The poposed newok could be exended o moe highe epesenaion of he wold. Fig. 4 shows he esul of he pinciple componen analysis of he acivaion paens in he hidden laye of he pedicion module (he image segmens ae and he pedicion ime sep is 5). This gaph shows he acivaion pa- PC siuaion siuaion 3 PC 5 siuaion 2 Figue 4: Pincipal componen analysis of he acivaion paens of hidden laye in pedicion module ens can be self-oganized depending on he siuaion. So, he infomaion of he acivaion paens can be used o discen he saes such as The ball is goes o lef on he hoizonal line.. This implies he possibiliy o consuc highe absac concep based on he self-oganizaion of lowe daa hough he boom-up appoach. Objec pemanence is hough o be closely elaed o memoy. To ealize an objec does no disappea behind an obsacle and will appea again, an agen should ecognize ha he eappeaed objec is he same one as he pevious one. In he poposed newok, if he aended objec disappeas behind some obsacle, he obo could no eain is aenion because he obo will elease is aenion based on he aenion level funcion 2. Howeve, if he pedicion module ha enables long em pedicion is available, he obo can eain is aenion and elae he objec behavio duing disappeaing and eappeaing. The key faculy fo his leaning is how long woking memoy can ecod he seies of evens and how he pedicion module will lean fom he evens in he woking memoy. In fac, i is epoed ha he woking memoy abiliy of infans is enhanced fom 7.5 monhs (2 secs) o 2 monhs (2 sec) (Schwaz and Reznick, 999) (Reznick e al., 24). We ae now conducing he expeimens o elae he pedicion abiliy of a disappeaed objec and he ime lengh of memoy. Refeences Baillageon, R. and DeVos, J. (99). Objec pemanence in young infans: fuhe evidence. Child developmen, 62: Baillageon, R., Spelke, E. S., and Wasseman, S. (985). Objec pemanence in five-monh-old infans. Cogniion, 2:9 28. Fizpaick, P., Needham, A., Naale, L., and Mea, G. (28). Shaed challenges in objec pecepion fo obos and infans. Jounal of Infan and Child Developmen, 7():7 24. Hinon, G. E. (27). Leaning muliple layes of epesenaion. TRENDS in Cogniive Sciences, (): Hinon, G. E., Osindeo, S., and Teh, Y.-W. (26). A fas leaning algoihm fo deep belief nes. Neual Compuaion, 8: Ii, L., Dhavale, N., and Pighin, F. (23). Realisic avaa eye end head animaion using a neuobiological model of visual aenion. Poceedings of SPIE. Knudsen, E. I. (27). Fundamenal componens of aenion. Annual Review of Neuoscience, 3: Lee, H., Ekanadham, C., and Ng, A. Y. (28). Spase deep belief ne model fo visual aea v2. In Poceedings of he Neual Infomaion Pocessing Sysems (NIPS) 2. Love, A. and Scasselai, B. (24). Using a obo o eexamine looking ime expeimens. In Poceedings of he 4h Inenaional Confeence on Developmen and Leaning (ICDL). Piage, J. (954). The consucion of ealiy in he child. basic books. Reznick, J. S., Moow, J. D., Goldman, B. D., and Snyde, J. (24). The onse of woking memoy in infans. Infancy, 6(). Schlesinge, M. (23). A lesson fom oboics: Modeling infans as auonomous agens. Adapive Behavio, (2). Schwaz, B. B. and Reznick, J. S. (999). Measuing infan spaial woking memoy using a modified delayed-esponse pocedue. Memoy, 7: 7.

Reinforcement learning

Reinforcement learning Lecue 3 Reinfocemen leaning Milos Hauskech milos@cs.pi.edu 539 Senno Squae Reinfocemen leaning We wan o lean he conol policy: : X A We see examples of x (bu oupus a ae no given) Insead of a we ge a feedback