HMM-Based Semantic Learning for a Mobile Robot

Size: px

Start display at page:

Download "HMM-Based Semantic Learning for a Mobile Robot"

Ralph Pierce
6 years ago
Views:

1 HMM-Based Sematic Learig for a Mobile Robot Kevi Squire Laguage Acquisitio ad Robotics Group Uiversity of Illiois at Urbaa-Champaig Adviser: Stephe E. Leviso

2 Laguage Learig Kevi Squire Licol Laboratory Iterview, 6 Jue 2005 p.1

3 Laguage Learig... by Robot! Kevi Squire Licol Laboratory Iterview, 6 Jue 2005 p.2

4 Philosophy of Laguage Acquisitio Fudametal Ideas: The Laguage Egie is primarily sematic, ot sytactic. There is o such thig as a disembodied mid. Laguage ad meaig is acquired through iteractio with the real world. Sesory-motor fuctio is essetial for huma-like cogitio. Metal processes are largely based o associative memory ad learig. Kevi Squire Licol Laboratory Iterview, 6 Jue 2005 p.3

5 Ph.D. Backgroud ad Research 1. Ifrastructure Developmet Hardware Software 2. Research Sematic Learig HMM Cascade Model Kevi Squire Licol Laboratory Iterview, 6 Jue 2005 p.4

6 Ph.D. Backgroud ad Research 1. Ifrastructure Developmet Hardware Software 2. Research Sematic Learig HMM Cascade Model Kevi Squire Licol Laboratory Iterview, 6 Jue 2005 p.4

7 Robot Hardware Kevi Squire Licol Laboratory Iterview, 6 Jue 2005 p.5

8 Robot Hardware Modificatios: Added cameras microphoes o-board computer wireless trasmitter Miscellaeous structural chages Replaced power supply, rewired to supply power to all compoets Istalled Liux Kevi Squire Licol Laboratory Iterview, 6 Jue 2005 p.6

9 Robot Hardware Modificatios: Added cameras microphoes o-board computer wireless trasmitter Miscellaeous structural chages Replaced power supply, rewired to supply power to all compoets Istalled Liux Kevi Squire Licol Laboratory Iterview, 6 Jue 2005 p.6

10 Robot Hardware Modificatios: Added cameras microphoes o-board computer wireless trasmitter Miscellaeous structural chages Replaced power supply, rewired to supply power to all compoets Istalled Liux Kevi Squire Licol Laboratory Iterview, 6 Jue 2005 p.6

11 Distributed Commuicatios Kevi Squire Licol Laboratory Iterview, 6 Jue 2005 p.7

12 Distributed Computig Framework Illy (robot) full (old) fillig Audio Rig Buffer full Soud Source Locatio (Sik) full etwork Audio Source (Remote) Hal (workstatio) Audio Source (Soud Card) Audio Server (Sik) fillig etwork Audio Server (Sik) full Audio Rig Buffer full full (old) Speech Recogitio (Sik) Kevi Squire Licol Laboratory Iterview, 6 Jue 2005 p.8

13 Distributed Computig Framework Allowed the itegratio of: Soud source localizatio (D. Li) Visio based avigatio & learig (W. Zhu) Speech recogitio (Q. Liu/R.S. Li) Simple workig memory (K. Squire) Next steps: Cetralized cotroller (M. McClai) Sematic learig (K. Squire) Kevi Squire Licol Laboratory Iterview, 6 Jue 2005 p.9

14 Ph.D. Backgroud ad Research 1. Ifrastructure Developmet Hardware Software 2. Research Sematic Learig HMM Cascade Model Kevi Squire Licol Laboratory Iterview, 6 Jue 2005 p.10

15 Ph.D. Backgroud ad Research 1. Ifrastructure Developmet Hardware Software 2. Research Sematic Learig HMM Cascade Model Kevi Squire Licol Laboratory Iterview, 6 Jue 2005 p.10

16 Cogitive Framework Outside World Sesory System Proprioceptive Feedback Motor System Noetic System Somatic System Kevi Squire Licol Laboratory Iterview, 6 Jue 2005 p.11

17 Associative Learig ad Memory Outside World Sesory System Proprioceptive Feedback Motor System Noetic System Somatic System Associative Memory Workig Memory Noetic System Kevi Squire Licol Laboratory Iterview, 6 Jue 2005 p.12

18 Associative Learig ad Memory Associative Memory Workig Memory Noetic System Associative Memory Procedural Memory Sematic Memory Episodic Memory Kevi Squire Licol Laboratory Iterview, 6 Jue 2005 p.13

19 Sematic Memory Associative (Log term) Memory Procedural Memory Sematic Memory Episodic Memory to Workig Memory Sematic Memory Cocept Model Visual Model Auditory Model Sesory Iputs Kevi Squire Licol Laboratory Iterview, 6 Jue 2005 p.14

20 Cocepts Cocept of Apple Other kowledge: facts, stories, experieces, etc. "Apple" * cruch * Cocept: abstract symbol associated with symbolic represetatios i the various seses Kevi Squire Licol Laboratory Iterview, 6 Jue 2005 p.15

21 Sematic Memory to Workig Memory Sematic Memory Cocept Model Visual Model Auditory Model Sesory Iputs Kevi Squire Licol Laboratory Iterview, 6 Jue 2005 p.16

22 Cascade of HMMs to Workig Memory ^ x co Sematic Memory Cocept Model ϕ^ co ϕ^ Visual Model Auditory Model = ϕ^ vis ^ x vis co ^ vis ^aud y ={x, x } ϕ^ aud x^ aud Sesory Iputs y vis y aud Kevi Squire Licol Laboratory Iterview, 6 Jue 2005 p.17

23 Hidde Markov Models (HMMs) S1 S2 S3 0.2 Kevi Squire Licol Laboratory Iterview, 6 Jue 2005 p.18

24 Hidde Markov Models (HMMs) S1 S2 S Kevi Squire Licol Laboratory Iterview, 6 Jue 2005 p.18

25 Hidde Markov Models (HMMs) 0.1 S S S3 0.6 Kevi Squire Licol Laboratory Iterview, 6 Jue 2005 p.18

26 Hidde Markov Models (HMMs) 0.1 S S S3 0.6 Kevi Squire Licol Laboratory Iterview, 6 Jue 2005 p.18

27 Maximum-Likelihood Estimatio (1) Traditioal methods (Baum-Welch reestimatio): Let p (y 1,..., y ; ϕ) be the likelihood of observatios (y 1,..., y ) give HMM ϕ. Maximize p (y 1,..., y ; ϕ) by solvig ϕ p (y 1,..., y ; ϕ) = 0. Implemeted as a Expectatio-Maximizatio (EM) procedure. Requires all of (y 1,..., y ) be available. Kevi Squire Licol Laboratory Iterview, 6 Jue 2005 p.19

28 Maximum-Likelihood Estimatio (2) Recursive (stochastic gradiet) procedure (LeGlad ad Mevel): Rewrite p (y 1,..., y ; ϕ) as a sum log p (y 1,..., y ; ϕ) = log b(y k ; ϕ) u k (ϕ) k=1 where b i (y k ; ϕ) = p(y k x k = i) u ki (ϕ) = Pr(x k = i y 1,..., y k 1 ). Update parameters ϕ at time + 1 with ϕ = ϕ + ε ( ϕ log b(y +1 ; ϕ) u +1 (ϕ) ) Kevi Squire Licol Laboratory Iterview, 6 Jue 2005 p.20

29 Cascade of HMMs ^ x co ϕ^ co ϕ^ ϕ^ vis ^ x vis co ^ vis ^aud y ={x, x } ϕ^ aud x^ aud y vis y aud Kevi Squire Licol Laboratory Iterview, 6 Jue 2005 p.21

30 Sematic Learig Simulatio Cocept Model Cocept Model Visual Model Auditory Model Visual Model Auditory Model "Apple" Kevi Squire Licol Laboratory Iterview, 6 Jue 2005 p.22

31 Sematic Learig Simulatio "Apple" Kevi Squire Licol Laboratory Iterview, 6 Jue 2005 p.22

32 Simulatio Model Topology ϕ _ boy ϕ c c ϕ^ robot ϕ^_ y = x c,1 ^ v c,2 y x ^v ^ c ^v y ={x, x ^ a } x ^a v ϕ a λ ϕ^v ϕ^a v y = y vis y a y vis y aud a = y y vis ϕ vis Kevi Squire Licol Laboratory Iterview, 6 Jue 2005 p.23

33 Simulatio Results Maximum Likelihood Classificatio Viterbi Classificatio ˆϕ a 90.1%(3.7%) 89.9%(4.2%) ˆϕ v 97.9%(1.6%) 99.1%(2.4%) ˆϕ c 98.4%(1.1%) 98.8%(1.1%) Average classificatio accuracy for the leared models over 50 rus. The umber i parethesis is the stadard deviatio. Kevi Squire Licol Laboratory Iterview, 6 Jue 2005 p.24

34 Robot Implemetatio The robot should: Recogize visual iputs Recogize auditory iputs Lear cocepts Kevi Squire Licol Laboratory Iterview, 6 Jue 2005 p.25

35 Cascade of HMMs as a Associative Memory Cascade Model: ^ x co ϕ^ co ϕ^ ϕ^ vis ^ x vis co ^ vis ^aud y ={x, x } ϕ^ aud x^ aud y vis y aud Kevi Squire Licol Laboratory Iterview, 6 Jue 2005 p.26

36 Cascade of HMMs as a Associative Memory Auditory-oly Classificatio: ^ x co ϕ^ co ϕ^ ϕ^ vis x^ vis co ^ aud y = x ϕ^ aud ^ x aud y vis y aud Kevi Squire Licol Laboratory Iterview, 6 Jue 2005 p.26

37 Cascade of HMMs as a Associative Memory Visual-oly Classificatio: ^ x co ϕ^ co ϕ^ ϕ^ vis ^ x vis co ^ vis y = x ϕ^ aud x^ aud y vis y aud Kevi Squire Licol Laboratory Iterview, 6 Jue 2005 p.26

38 Cascade of HMMs as a Associative Memory Audio-Visual Learig: ^ x co ϕ^ co ϕ^ ϕ^ vis ^ x vis co ^ vis ^aud y ={x, x } ϕ^ aud x^ aud y vis y aud Kevi Squire Licol Laboratory Iterview, 6 Jue 2005 p.26

39 Robot Demostratio Visual Objects: Words: cat dog red ball gree ball ball aimal Cocepts: cat dog red ball gree ball ball aimal Kevi Squire Licol Laboratory Iterview, 6 Jue 2005 p.27

40 Robotic Cotroller speech: "illy"/ tur toward soud / explore 1 Explore visible object object far/ approach object lear object ukow speech/ beep 6 Iteract hear kow object/ search for object silece (timeout) timeout expired/ beep / ru ito object lost object 2 Foud Object object ear 3 Lear Name speech/ repeat & lear foud desired object/ say ame 7 Search 5 Play 2 silece (timeout) foud othig or wrog object/ explore / pick up object 4 Play 1 Kevi Squire Licol Laboratory Iterview, 6 Jue 2005 p.28

41 Video Kevi Squire Licol Laboratory Iterview, 6 Jue 2005 p.29

42 Coclusio 1. Built a platform upo which to coduct laguage acquisitio research. 2. Proposed a geeral model of sematic cocept learig. 3. Successfully implemeted this model i a real robot usig a cascade of HMMs. Kevi Squire Licol Laboratory Iterview, 6 Jue 2005 p.30

43 Future Directios/Iterests 1. Grow/shrik models (combie/split states) as appropriate (e.g., use miimum descriptio legth (MDL) or related measures). 2. Apply associative learig to spatial/actio iformatio, other modalities. 3. Study setece comprehesio (e.g., combie sytactic ad sematic iformatio). 4. Icorporate reiforcemet ito curret usupervised traiig scheme. Kevi Squire Licol Laboratory Iterview, 6 Jue 2005 p.31

44 Questios? Kevi Squire Licol Laboratory Iterview, 6 Jue 2005 p.32

45 Robot Visual HMM Iitializatio: Kmeas o labeled data Olie Learig with RMLE ϕ^ obj S1 S2 ^ obj x S3 Features: Color histogram Momet Height/width ratio ~ obj y Fixed umber of classes y obj Kevi Squire Licol Laboratory Iterview, 6 Jue 2005 p.33

46 Robot Auditory Model 3 State HMM States: Silece, Voiced, Uvoiced Features: 3 log-area ratios + log-eergy + voicig Olie update with RMLE possible ^ word x ϕ^ word S1 S2 S3 ~ word y Word Recogizer Features: histogram of audio states + word legth Ca distiguish some words Iitial traiig offlie with RMLE ϕ^ aud aud {y } ^aud {x } Kevi Squire Licol Laboratory Iterview, 6 Jue 2005 p.34

47 Robot Cocept HMM 6 State Discrete HMM Observatios: states of Visual ad Auditory models S1 S2 S3 Iitialized offlie Olie update with RMLE Kevi Squire Licol Laboratory Iterview, 6 Jue 2005 p.35

48 RMLE Covergece (1) Coditios for RMLE Covergece: The trasitio probability matrix A(ϕ ) is aperiodic ad irreducible. The mappig ϕ A(ϕ) is twice differetiable with bouded first ad secod derivatives ad Lipschitz cotiuous secod derivative. The mappig ϕ b(y k ; ϕ) is three times differetiable, ad the fuctio b(y k ; θ) is cotiuous o R for every θ Θ. Alterately, for y k draw from a fiite alphabet, the mappig ϕ b(y k ; ϕ) is twice differetiable with bouded first ad secod derivatives ad Lipschitz cotiuous secod derivative. Kevi Squire Licol Laboratory Iterview, 6 Jue 2005 p.36

49 RMLE Covergece (2) Coditios for RMLE Covergece: Uder P ϕ, the exteded Markov chai {X, Y, u (ϕ), w (ϕ)} is geometrically ergodic (see LeGlad ad Mevel, 96). Because of this geometric ergodicity, the iitial values of u 0 (ϕ) ad w 0 (ϕ) are forgotte expoetially fast. Kevi Squire Licol Laboratory Iterview, 6 Jue 2005 p.37

50 MPEG Video Kevi Squire Licol Laboratory Iterview, 6 Jue 2005 p.38

Expectation-Maximization Algorithm.

Expectation-Maximization Algorithm. Expectatio-Maximizatio Algorithm. Petr Pošík Czech Techical Uiversity i Prague Faculty of Electrical Egieerig Dept. of Cyberetics MLE 2 Likelihood.........................................................................................................