Ling 3701H / Psyh 3371H: Leture Notes 9 Hierrhi Sequentil Predition Contents 9.1 Complex events.................................... 1 9.2 Reognition of omplex events using event frgments................ 2 9.3 Reognition Model.................................. 2 9.4 Exmple reognition y hierrhi sequentil predition.............. 4 9.5 Prtie........................................ 6 Lnguge proessing my e sed on domin-generl omplex event predition. This uses memory nd generliztion (lerning) to reognize omplex events (plns). (Rell tht events my e represented in the rin s elementry preditions. We will ssume events re lso onneted y elementry preditions of ustion.) 9.1 Complex events Events n ontin hierrhies of suevents, espeilly omplex plns (omplex ides): nts-in-mouth nts-on-stik stik-in-mouth nts-in-nthill stik-in-nthill stik-in-fingers fingers-in-nthill rnh-in-fingers...... rnh-in-fingers twig-wy-from-rnh hnd-on-twig hnd-wy-from-rnh Su-events re relted to prent events y use elementry preditions. When similr (reognition) opertions re nested inside other opertions, proess is lled reursive. 1
9.2 Reognition of omplex events using event frgments Humns nd (some) nimls n reognize nd re-rete omplex hierrhi events. [Fuster, 1990, otvinik, 2007] Prtil sequenes of events n e grouped nd stored s event frgments /, where: is predited whole pex top-level event or su-event, is n expeted prt se su-event / oserved event yet to ome, whih ompletes the whole. E.g. nts-in-nthill n e ounted s nts-on-stik/stik-in-nthill. Use ued ssoition ( ) to diretly link n individul expettion to supported predition. Ner-omplete su-events n e hined together to sve memory: E.g. nts-on-stik/stik-in-nthill nd stik-in-fingers form nts-on-stik/fingers-in-nthill. When reent event frgment is ompleted, it n e dded to n erlier event frgment. E.g. if stik-in-fingers is omplete, it n stisfy stik-in-nthill with fingers-in-nthill expeted. Use ued ssoition ( ) to diretly link n individul predition to preeding expettion. Unertinty out events my e modeled using superposed tivtion vetors, desried erlier. 9.3 Reognition Model This model mintins sequene of event frgments essile from the urrent expettion : E.g. / is nts-on-stik/stik-in-nthill, / is stik-in-fingers/twig-wy-from-rnh. Cruilly, this store n only e few elements long efore interferene uses troule. The model lso ssumes set of lerned predition rules: 2
E.g. nts-on-stik () is omposed of nts-in-nthill () followed y stik-in-nthill (). Here,,, nd might e onneted y use elementry predition (mgent lines). First, distinguish frontier (simple, oserved) nd juntion (omplex, hidden) events: nts-in-mouth nts-on-stik stik-in-mouth nts-in-nthill stik-in-nthill stik-in-fingers fingers-in-nthill rnh-in-fingers...... rnh-in-fingers twig-wy-from-rnh hnd-on-twig hnd-wy-from-rnh Complex ides n now e ssemled y onneting oserved events to event frgments... Frontier deision (dd oserved event nd onnet to existing event frgment, or don t): Yes-mth outome (set urrent predition): No-mth outome (hek types, store ued ssoition from to, set urrent predition): Juntion deision (pply predition rule nd onnet resulting event frgment, or don t): 3
Yes-mth outome (hek types, pply rule, store ued ssoition from to ): No-mth outome (pply rule, store ued ssoition from to nd to : Mthing n e implemented in simple neurl networks, generlized y proedurl lerning. These opertions n reognize ny rnhing event struture using minimum mount of memory. 9.4 Exmple reognition y hierrhi sequentil predition Here is n exmple of reognizing omplex pln from oservtions. Strt with oservtion of nthill, predit nts on stik 1, nd expet stik in the nthill 1 : nts-in-nthill 1 nts-on-stik 1 stik-in-nthill Perhps other preditions nd expettions, like pushing over the nthill, re lso superposed t 1. 4
Then fork oservtion of rnh 2, predit stik 2 ; don t join, leving new event frgment: nts-in-nthill 1 nts-on-stik 2 rnh-in-fingers 2 stik-in-fingers 1 stik-in-nthill 2 twig-wy-from-rnh Then fork off oservtion out gring twig 3, nd join it to the previous event frgment: nts-in-nthill 1 nts-on-stik 2 1 stik-in-nthill 2 stik-in-fingers 3 twig-wy-from-rnh rnh-in-fingers 3 hnd-on-twig 3 hnd-wy-from-rnh Then don t fork oservtion 4 to omplete 2, nd join it to the previous event frgment, leving only one event frgment: nts-in-nthill 2 1 nts-on-stik 2 rnh-in-fingers 3 hnd-on-twig 4 stik-in-nthill stik-in-fingers 3 twig-wy-from-rnh 4 hnd-wy-from-rnh 4 fingers-in-nthill 5
Then don t fork fingers t nthill 5, predit nts in mouth 5, expet stik in mouth 5 : nts-in-nthill 2 1 nts-on-stik 2 rnh-in-fingers 3 hnd-on-twig 4 stik-in-nthill stik-in-fingers 3 twig-wy-from-rnh 5 nts-in-mouth 4 hnd-wy-from-rnh 5 fingers-in-nthill 5 stik-in-mouth The struture of rule pplitions over time n e drwn s tree: 9.5 Prtie ssume the following omplex event is eing reognized: seed-on-rok nut-on-rok...... stone-in-hnd stone-to-nut hnd-to-nut nd the following event frgments exist fter the oservtion of nut-on-rok: nut-on-rok 1 seed-on-rok 1 stone-to-nut Drw the event frgments tht would exist fter the oservtion of stone-in-hnd. Whih opertions on frontier nd juntion nodes re used to proess this oservtion? 6
Referenes [otvinik, 2007] otvinik, M. (2007). Multilevel struture in ehvior nd in the rin: omputtionl model of Fuster s hierrhy. Philosophil Trnstions of the Royl Soiety, Series : iologil Sienes, 362:1615 1626. [Fuster, 1990] Fuster, J. M. (1990). ehviorl eletrophysiology of the prefrontl ortex of the primte. Progress in rin Reserh, 85:313 324. 7