Pattern recognition for chemical sensor arrays with neuromorphic models of the olfactory system

Size: px

Start display at page:

Download "Pattern recognition for chemical sensor arrays with neuromorphic models of the olfactory system"

Tobias Wade
5 years ago
Views:

1 Pattern recognition for chemical sensor arrays with neuromorphic models of the olfactory system Pattern Recognition and Intelligent Sensor Machines Lab Department of Computer Science Texas A&M University 1

2 Outline g Introduction g Combinatorial coding g Chemotopic convergence g Contrast enhancement g Habituation dynamics g Cortical feedback g Discussion 2

3 Acknowledgments g Lab members n Nilesh Powar n Barani Raman n Agustin Gutierrez-Galvez n Alex Perera n Takao Yamanaka g Funding n National Science Foundation (CAREER Award ) 3

4 g Introduction g Combinatorial coding g Chemotopic convergence g Contrast enhancement g Habituation dynamics g Cortical feedback g Discussion 4

5 Motivation for this work g A neuromorphic systems perspective n Biology serves as a source of inspiration to solve engineering problems g g As engineers, we would be foolish to ignore the lessons of a billion years of evolution Carver Mead, 1993 Already exploited in other domains: computer vision, speech processing, legged locomotion, behavior n Can engineering systems be used as models to test biological/cognitive hypotheses? [Webb, 2000] g Long-term research objective n To develop novel algorithms for chemical sensor arrays inspired by information processing in the biological olfactory system 5

Human olfactory anatomy g Olfactory epithelium n Primary reception g Olfactory receptor neurons g Diversity and redundancy OLFACTORY BULB CORTEX LIMBIC SYSTEM g Olfactory bulb n Signal processing g

6 Human olfactory anatomy g Olfactory epithelium n Primary reception g Olfactory receptor neurons g Diversity and redundancy OLFACTORY BULB CORTEX LIMBIC SYSTEM g Olfactory bulb n Signal processing g Convergence g Lateral inhibitory circuits OLFACTORY EPITHELIUM ODOR PARTICLES OLFACTORY TRACT g Olfactory cortex n Storage and association g Piriform cortex (recognition) g Limbic system (emotions) ODOR SIGNALS OLFACTORY RECEPTOR NEURONS CILIA ODOR MOLECULES 6

7 A computational view of olfaction I. Olfactory epithelium 1 ORN Combinatorial coding Chemotopic convergence 2 II. Olfactory Bulb Volume control Contrast enhancement 3 Holistic perception Modulatory feedback 4 LEGEND Adapted from [Mori et al., 1999] 6 III. Olfactory Cortex 5 ORN Olfactory receptor neuron GL Glomerulus PG Periglomerular interneuron M Mitral neuron T Tufted neuron GR Granule interneuron 7

8 g Introduction g Combinatorial coding g Chemotopic convergence g Contrast enhancement g Habituation dynamics g Cortical feedback g Discussion 8

9 Combinatorial coding (bio vs. e-noses) g Dimensionality mismatch n 100M ORNs in the (human) olfactory system n A handful of sensors in electronic noses g How do we achieve high-dimensionality? n Micro-bead arrays (Illumina, McDevitt, ) n Analytical instruments (FTIR, GCMS, IMS ) n Sensor modulation approaches 9

10 Temperature modulation in MOS g Selectivity of MOS layers depends on the operating temperature n Isothermal control: maintain constant temperature n Temperature modulation: capture sensor response while changing temperature [Yamazoe and Miura, 1992] 10

11 Temperature modulation patterns V H time 1/R S time x Hz E D B C A B C D E AIR B Acetone ACETONE C Ammonia AMMONIA D IPA E IPA VINEGAR A A Air Acetic Acid 11

12 g Introduction g Combinatorial coding g Chemotopic convergence g Contrast enhancement g Habituation dynamics g Cortical feedback g Discussion 12

13 ORN projection onto olfactory bulb g Chemotopic convergence n ORNs expressing the same receptor gene project onto one or a few GL n GL serve as labeled lines or odotope detectors ORN GL from [Yokoi et al., 1995] M/T P2 receptor; mouse OB [Bulfone et al., 1998] 13

14 Glomerular coding (in biology) g Quality coding n 18 amino acids (AA), zebrafish olfactory bulb g Intensity coding n 5 AA concentrations [Friedrich and Korsching, 1997] 14

15 Glomerular coding (in biology) g Quality coding n In-vivo recording n Honeybee antennal lobe [Joerges et al., 1995] 15

16 Defining sensor selectivity g Affinity (or class) space n Sensor response across a collection of odors i [ ] O O S, S,...,S 1 2 S = i n Complementary to feature space: odor response across a collection of sensors i O C i FEATURE SPACE AFFINITY SPACE S2 A A A A A A A A A A A A BB B B B B B B B B B B S1 S1 S1 S1 S1 S1 S1 S2 S2 S2 S2 S2 S2 S2 S1 A 16

17 Sensor chemotopic convergence (1) g Convergence n Cluster sensors in affinity space (topographically) n Compute average activation at each cluster n The averaging process reduces sensor noise 17

18 Sensor chemotopic convergence (2) g Simulation results n 400,000 ORN, 400 GL, 10 ligands L 1 L 2 L 3 L 4 L 5 L 6 L 7 L 8 L 9 L 10 L 1 L 1 +L 2 logc= [Gutierrez-Osuna, 2002] 18

19 Experimental data g Sensors n TGS 2600, TGS 2620 n 0-7V heater voltage; 2.5min period n Static headspace g Analytes n Acetone, isopropyl alcohol and ammonia n Baseline conc.: 0.3 v/v%, 1.0 v/v% 33 v/v% resp. n Dilutions: 2 additional with 1/3 dilution factor n No. samples: 3 analytes 3 conc. 3 days + neutrals 19

20 Experimental TM responses 1 Sensor 1 Sensor 2 Sensor conductance (normalized) Ammonia Isopropyl Alcohol Acetone Pseudo-sensors 20

21 MOS affinity space and SOM Sensor SOM node SOM lattice Ammonia Acetone Isopropyl alcohol 21

22 Sensor chemotopic convergence (3) C A B

23 Convergence coding (in sensors) ACETONE ISOPROPYL ALCOHOL AMMONIA Higher concentrations 23

24 g Introduction g Combinatorial coding g Chemotopic convergence g Contrast enhancement g Habituation dynamics g Cortical feedback g Discussion 24

25 Contrast enhancement g The role of mitral-granule interactions n Sharpening of the individual molecular tuning range of mitral cells with respect to GL A A B C ORN GL B C n-cho n-cho D GR M/T n-cho D (+) (-) n-cho from [Yokoi et al., 1995] 25

26 Model of bulbar circuits g Based on Grossberg s additive model dm j dt ( t) m ( t) ( m ( t) ) M j = + Lkjϕ k τ j k = neuron lateral dynamics inhibition + 43 G j { ORN input Impulse response δ(t) G 1 G 2 G 3 G N t m 1 m 2 m 3 m N? m j (t) L kj t 26

27 L kj : Lateral interactions g Center on-off surround n Excitatory connections with local neurons n Inhibitory connections with distant neurons g Biological relevance n Retinal receptive fields mediated by ganglion cells n Recently observed in OB [Angust et al., 2003] 27

28 Odor-driven attractor dynamics (antennal lobe, honeybee) From [Galan et al., 2003] 28

29 Lateral interaction results (in sensors) Higher concentrations Center Surround SOM ACETONE 1 3 ISOPROPYL ALCOHOL AMMONIA

30 g Introduction g Combinatorial coding g Chemotopic convergence g Contrast enhancement g Habituation dynamics g Cortical feedback g Discussion 30

31 Habituation g A decrease in the strength of a behavioral response with repeated stimulation n Spike frequency adaptation: The frequency of action potentials (firing rate) of a neuron slows down over time 31

32 Modeling habituation g Implemented by means of an inhibitory term h j (t) in the additive model dm A j dt ( t) A m M j ( t) = + Lkjϕ k j τ k = 1 k j Grossberg' s additive model (same as before) ( A ) A m ( t) + I h () t j { Habituatio term n n where h j (t) is governed by a moving average h j ( t) t = α m ( τ ) exp 0 j [ β ( t τ )] dτ 32

33 Habituation trigger (convolution) h j ( t) t = α m ( τ ) exp 0 j [ β ( t τ )] dτ m j (t) exp(-β(t-τ)) t h j (t) t t 33

34 Odor-driven attractor dynamics 4 2 Ammonia Trajectories originate close to each other PC PC2 2 Acetone Isopropyl alcohol 0 PC

35 Emerging dynamic code Dynamics without habituation fixed-point attractor Dynamics with habituation limit cycle attractor 4 2 Ammonia Trajectories originate close to each other PC PC2 2 Acetone 4-2 Isopropyl alcohol 0 PC Question: Does the limit cycle contain additional discriminatory information? 35

36 Emerging temporal code Region A B C (A) ACETONE 1 3 Region C A B (B) IPA Region A C B (C) AMMONIA Region A 0.2 B 0.1 C Region C A B

37 g Introduction g Combinatorial coding g Chemotopic convergence g Contrast enhancement g Habituation dynamics g Cortical feedback Cortical feedback g Discussion 37

38 Bulbar-cortical interaction (1) g Bulb-to-cortex projection n Non-topographic, many-to-many n Cortical neurons serve as coincidence detectors n Analytical reception vs. synthetic perception from [Wilson and Stevenson, 2003] 38

39 Bulbar-cortical interaction (2) g Cortex-to-bulb feedback n Primarily onto inhibitory granule interneurons n Dense but unknown connectivity g Functional hypotheses n Segmentation of odor mixtures [Li and Herz, 2000] n Filtering bulbar inputs / resonance [Grossberg, 1976] n Chaotic dynamics [Yao and Freeman, 1990] n Hierarchical clustering [Ambros-Ingerson et al., 1990] 39

40 Bulbar-cortical model (1) g Two densely-connected additive models n OB-OB : contrast enhancement n OC-OC : associate memory n OB OC projection: convergent/divergent n OC OB feedback:???? G1 G2 G3 GN m1 m2 FF ji m 3 L kj mn M 1 = M + Lϕ 14 τ (same as before) ( M ) + G + FBϕ( P) Cortex feedback to bulb p1 p2 p 3 AC ki pn FB ij P 1 = P + ACϕ 14 λ Cortico-cortical connections ( P) + FFϕ( M ) Feedforward connections 40

41 Learning feedback connections g Two rules two functions n Anti-hebbian FB = γ ( PM T ) g g Feedback inhibits bulb activity responsible for the cortical response Results in temporal segmentation of mixtures n Hebbian FB = γ ( PM T ) g g Feedback inhibits bulb neurons other than those responsible for the cortical response System locks onto a particular odor, suppressing the background/weaker odor 41

42 Experimental results g Model structure n n OB with 6 units OC with 6 units (manually labeled) g Learning n n n FF, AC connections: Hebbian Training: pure compounds Testing: mixtures Sensor Conductance (L1-normalized) Iso-propyl alcohol (I) Acetone (A) Ammonia (M) Mixture (AIM) Pseudo-sensors Segmentation with anti-hebbian feedback Odor suppression with Hebbian feedback Bulb activity Cortex activity Bulb activity Cortex activity B1 (A) C1 B1 (A) C1 B2 (A) C2 B2 (A) C2 B3 (I) C3 B3 C3 B4 B5 (I) C4 (M) C5 Presence of ammonia B4 B5 C4 (M) C5 B6 (M) C6 B6 (M) C6 42

43 g Introduction g Combinatorial coding g Chemotopic convergence g Contrast enhancement g Habituation dynamics g Cortical feedback g Discussion 43

44 Discussion (1) g At what level of abstraction should we mimic biology? n Statistical, perceptron, Hopfield, KIII, I&F... g Marr s (1982) levels of analysis n Computational theory g What is the goal of the computation? n Representation and algorithm g What is the representation for the input and output, and what is the algorithm for the transformation? n Hardware implementation g How can the representation and algorithm be realized physically? 44

45 Adaptation as a subspace projection g Project feature vector x onto subspace that minimizes effects from previous stimulus ω Input space Projection matrix Adapted space Class label x W(ω) y ω Cancellation signal From [Gutierrez-Osuna and Raman, 2005] 45

46 Adaptation results (pure compound) 46

47 Adaptation results (mixture) 47

48 Discussion (2) g When does neuromorphic make sense? n n With massively combinatorial AND redundant inputs And even better when inputs are compatible with the weak shape theory g But also to help pose new sensing problems, for which a simple implementation can be abstracted n n n n n n Background suppression Novelty detection Perceptual grouping vs. contrast enhancement Quality vs. intensity coding Habituation vs. selective attention Drift compensation 48

49 Discussion (3) g Open questions n To what extent is biological hardware a good match for the characteristics of chemical sensor arrays? n What level of abstraction is optimal? n Should we expect neuromorphic algorithms to outperform statistical techniques? 49

50 References g g g g g g g g g g Ambros-Ingerson, J., Granger, R. & Lynch, G. (1990). Simulation of paleocortex performs hierarchical clustering. Science, 247, Amrani, M. E. H., Persaud, K.C., & and Payne, P. A. (1995). High-frequency measurements of conducting polymers: development of a new technique for sensing volatile chemicals. Measurement Science and Technology 6(10), Friedrich, R.W., & Korsching, S.I. (1997). Combinatorial and chemotopic odorant coding in the zebrafish olfactory bulb visualized by optical imaging. Neuron, 18, Fuss, S. H., & Korsching, S. I. (2001). Odorant Feature Detection: Activity Mapping of Structure Response Relationships in the Zebrafish Olfactory Bulb. The Journal of Neuroscience, 21(21), Grossberg, S. (1976). Adaptive Pattern Classification and Universal Recording: II. Feedback, Expectation, Olfaction and Illusion. Biological Cybernetics, 23, Gutierrez-Osuna, R., & Nagle, H. T. (1999). A method for evaluating datapreprocessing techniques for odor classification with an array of gas sensors. IEEE Transactions on Systems, Man, and Cybernetics B, 29(5), Gutierrez-Osuna, R. (2002). A Self-organizing Model of Chemotopic Convergence for Olfactory Coding. In: Proc. 2nd Joint EMBS-BMES Conf. (23-26), Houston, TX. Ikohura, K., & Watson, J. (1994). The stannic oxide gas sensor. Principles and applications, CRC Press. Joerges, J., Kuttner, A., Galizia, C. G., & Menzel, R. (1997). Representations of odours and odour mixtures visualized in the honeybee brain. Nature, 387, Kunugi, Y., Nigorikawa, K., Harima, Y., & Yamashita, K. (1994). A selective organic vapour sensor based on simultaneous measurements of change of mass and resistance of a Poly(pyrrole) thin film. J. Chem. Soc. Chem. Commun.,

51 References g Lee, A. P., & Reedy, B. J. (1999). Temperature modulation in semiconductor gas sensing. Sensors and Actuators B 60, g Li, Z., and Hertz, J. (2000). Odor recognition and segmentation by a model olfactory bulb and cortex. Network: Computation in Neural Systems 11, g Nakamoto, T., Sukegawa, K., & Sumitomo, E. (2002). Higher-Order Sensing Using QCM Sensor Array and Preconcentrator with Variable Temperature. In: Proc. first IEEE international conference on Sensors, ( ), Orlando, FL. g Nakamoto,T., Isaka, Y., Ishige, T., & Moriizumi, T. (2000) Odor-sensing system using preconcentrator with variable temperature. Sensors and Actuators B, 69, g Otagawa, T., & Stetter, J.R. (1987). A chemical concentration modulation sensor for selective detection of airborne chemicals. Sensors and Actuators 11, 251. g Schaffer, R. E., Rose-Pehrsson, S. L., & McGill, A. (1993). Multiway analysis of preconcentrator-sampled surface acoustic wave chemical sensor array data. Field Analytical Chemistry and Technology 2(3), g Wang, D. (1998). Habituation. In: M.A. Arbib (Ed.), The Handbook of Brain Theory and Neural Networks, ( ), MIT Press. g Webb, B. (2000). What does robotics offer animal behavior. Animal Behavior, 60, g Weimar, U., & Gopel, W. (1995). A.C. measurements on tin oxide sensors to improve selectivities and sensitivities. Sensors and Actuators B 26-27, g Wilson, D. A., & Stevenson, R. J. (2003). The fundamental role of memory in olfactory perception. Trends in Neurosciences, 26(5), g Yao, Y., and Freeman, W. J. (1990). Model of Biological Pattern Recognition with Spatially Chaotic Dynamics. Neural Networks, 3, g Yokoi, M., Mori, K. & Nakanishi, S. (1995). Refinement of Odor Molecule Tuning by Dendrodendritic Synaptic Inhibition in the Olfactory Bulb, PNAS 92,

52 QUESTIONS 52

53 THANK YOU 53

arxiv:physics/ v1 [physics.bio-ph] 19 Feb 1999

arxiv:physics/ v1 [physics.bio-ph] 19 Feb 1999 Odor recognition and segmentation by coupled olfactory bulb and cortical networks arxiv:physics/9902052v1 [physics.bioph] 19 Feb 1999 Abstract Zhaoping Li a,1 John Hertz b a CBCL, MIT, Cambridge MA 02139