The functional organization of the visual cortex in primates Dominated by LGN M-cell input Drosal stream for motion perception & spatial localization V5 LIP/7a V2 V4 IT Ventral stream for object recognition & discrimination Dominated by LGN P- & K-cell input
Understanding how visual information is processed in the brain Three stages of neural processing 1. Low-level processing: First-order stimulus feature information (stimulus contrast, orientation, spatial frequency, temporal frequency, wavelength) is processed. 2. Mid-level processing: Grouping or linking of first-order stimulus information. 3. High-level processing: Decision-making mechanisms: Initiation of perceptual responses, e.g., recognition of an object in visual scenes.
1. Topographic organization of V1 2. Cell classification--simple vs. complex, & hypercomplex? 3. Orientation tuning-feed forward vs. long-range signals 4. Direction selectivity 5. Spatial / temporal frequency tuning 6. Ocular dominance 7. Functional organization (1) Ocular dominance columns 8. Functional organization (2) Orientation columns
Stimulus flashed ON Orientation Selectivity Spike trains for different stimulus orientations. Orientation tuning is one of the biggest differences between LGN & cortex. Orientation selectivity is an emergent (i.e., not present in the LGN) & invariant property (i.e. it is not qualitatively altered by other stimulus manipulations). For example, a cell will typically exhibit the same preferred orientation for a bright vs. a dark bar. From Hubel & Wiesel, 1962
Orientation Selectivity Examples of orientation tuning functions for three representative neurons. Orientation tuning is shown by the great majority of cortical cells and by all functional classes of cortical neurons. The preferred orientation and sharpness of orientation tuning are unaffected by stimulus contrast, when gratings were used as stimuli.
Orientation/Direction Selectivity Polar plots of orientation tuning for 4 representative neurons. The angle represents the stimulus orientation & the radial dimension is the response in spikes/sec. Drifting gratings From DeValois, 1982
Mechanisms of orientation selectivity Simple Cell LGN RFs Simple cells produced by converging inputs from a row of LGN that have the same RF polarity.
Simple Cell Model ON LGN centers make up the ON subregions in simple RFs. The OFF subregions reflect the centers of OFF center LGN cells (the LGN surround are not very obvious at cortex). The degree of orientation tuning will vary with the number of LGN inputs and the orderliness of their spatial retinotopy. From Wandel, 1995
Long-range & local connections affects orientation selectivity Local Orientation columns Long-range Bosking et al, 1997 From Senpigel & Blakemore, 1966
1. Topographic organization of V1 2. Cell classification--simple vs. complex, & hypercomplex? 3. Orientation tuning-feed forward vs. long-range signals 4. Direction selectivity 5. Spatial frequency tuning 6. Ocular dominance 7. Functional organization (1) Ocular dominance columns 8. Functional organization (2) Orientation columns
Direction Selectivity Spike trains for different stimulus orientations and different directions of movement. This cell is selective for both stimulus orientation and the direction of movement. From Hubel & Wiesel, 1962
Direction Selectivity Distribution of directional selectivity for a population of cortical cells. The direction ratio = the response in the non-preferred direction / response in the preferred direction. The bimodal nature of the distribution suggests that direction selective cells for a distinct subgroup. RF Preferred Non-Preferred 100 spikes/sec 10 spikes/ sec From DeValois
Linear summation model Bright bar
Directional Selectivity: Simple Cell Evidence for nonlinear contributions to directional selectivity Bicuculine blocks GABA mediated inhibition & as shown here eliminates directional selectivity in some cells. From Sillito, 1977
1. Topographic organization of V1 2. Cell classification--simple vs. complex, & hypercomplex? 3. Orientation tuning-feed forward vs. long-range signals 4. Direction selectivity 5. Spatial / temporal frequency tuning 6. Ocular dominance 7. Functional organization (1) Ocular dominance columns 8. Functional organization (2) Orientation columns
Spatial Frequency M M Magno vs. Parvo P P Response P Contrast Sensitivity Response (spikes/sec) M P M Contrast (%)
Spatial Frequency Tuning Examples of SF tuning functions for 6 cells recorded in a single penetration. The RFs were superimposed. Note the that at a given eccentricity the peak SF cover a wide range of SFs. Contrast sensitivity 1 10 Spatial Frequency (c/d) CSF is determined by the sensitivity of multiple spatial channels (V1neurons?) Contrast sensitivity 1 10 Spatial Frequency (c/d) Human Vision Cortical Neurons
Contrast Sensitivity Contrast Sensitivity Temporal Frequency Tuning P cells M cells Temporal frequency (Hz) Temporal frequency (Hz) Visual Latency: Integration time 55 msec 50 msec Movshon et al, 2005
1. Topographic organization of V1 2. Cell classification--simple vs. complex, & hypercomplex? 3. Orientation tuning-feed forward vs. long-range signals 4. Direction selectivity 5. Spatial/Temporal frequency tuning 6. Ocular dominance 7. Functional organization (1) Ocular dominance columns 8. Functional organization (2) Orientation columns
1. Topographic organization of V1 2. Cell classification--simple vs. complex, & hypercomplex? 3. Orientation tuning-feed forward vs. long-range signals 4. Direction selectivity 5. Spatial frequency tuning 6. Ocular dominance 7. Functional organization (1) Ocular dominance columns 8. Functional organization (2) Orientation columns
Ocular dominance columns in V1 R Long-range horizontal connections LGN I L Inject radioactive proline LGN VI V IVc II - III & IVAB Higher Order Visual Areas (Extrastriate visual areas)
Ocular Dominance Columns cross section white matter tangential section through layer IV layer IV From Hubel et al., 1978 Dark-field autoradiographs. Radioactive proline injected into one eye. The light areas represent parts of cortex that received inputs from the injected eye.
Response (spikes/sec) L Ocular Dominance Ocular dominance = relative ability of the 2 eyes to excite a cortical neuron. R Right V1 LGN Monkey V1 0.4 N = 238 0.3 0.2 0.1 Proportion of Units(%) 0 1 2 3 4 5 6 7 Contralateral Ipsilateral Ocular Dominance II-III IVC V-VI L R L R L
1. Topographic organization of V1 2. Cell classification--simple vs. complex, & hypercomplex? 3. Orientation tuning-feed forward vs. long-range signals 4. Direction selectivity 5. Spatial/Temporal frequency tuning 6. Ocular dominance 7. Functional organization (1) Ocular dominance columns 8. Functional organization (2) Orientation columns
Orientation Columns from Hubel et al., 1978 Preferred orientations for cells in 3 penetrations. Note that the preferred orientations are similar through the full thickness of cortex -- i.e. orientation columns extent from layer 1 to layer 6.
Orientation Columns layer IV orientation columns Deoxyglucose autoradiograpy of horizontal section through operculum. Staining pattern was produced by vertical moving strips presented to both eyes. The dark areas represent activated neurons. from Hubel et al., 1978
Orientation Columns Color coded map of the preferred orientations across the brain s surface obtained using a method to optically image brain activity. Pinwheel = area of converge of different orientation columns ( singularity ). From Blasdel, 1992
Relationship between CO Blobs, Orientation Columns & Ocular Dominance Columns Overlay of orientation & ocular dominance columns. The expected positions of CO blobs have been drawn. From Blasdel, 1992
Hypercolumn CO Blobs Hubel and Wiesel, 1982
Neural Basis of Visual Perception Neural Detectors & Network Dorsal Stream of the Extrastriate Visual Areas Primary Visual Cortex (Area 17, V1, Striate Cortex) Ventral Stream of the Extrastriate Visual Areas Network of Neurons Neurons Retina LGN Visual Cortex
Three stages of cortical processing 1. Low-level: First-order stimulus feature information (orientation, SF, TF) is processed: V1 2. Mid-level: Grouping or linking of first-order stimulus information: Low to mid-level extrastriate visual areas (V2, V3, V4, MT, & MST). 3. High-level: Initiation of perceptual responses, e.g., recognition of an object in visual scenes: Higher-centers (IT, LIP, beyond).
Ability to integrate local information over a large area RF of V1 neurons are too small to see the circle. V4 neurons can integrate (link) preferred orientations of Gabor patches (Curvature detectors). IT neurons can decode information from V4 -----> Seeing a circle (shape). 7 deg 1.6 deg
The size of RF increases with retinal eccentricities and cortical areas in the streams /V4 /V3 (V5) Bullier et al, 2003
Hierarchical Organization The large number of interconnections probably are important for coordinating processing between parallel streams and dynamic regulation of information flow. from Van Essen et al., 1992
Functional Streams Visual system separates different types of information in to parallel, anatomically segregated processing streams. Generally primate and human brains are considered to have 2 dominant streams 1) the occipitotemporal pathway, the ventral stream, which is thought to be crucial for object identification (the what pathway) & 2) the occipitoparietal pathway, the dorsal stream, which is thought to be crucial for spatial relationships & visual guidance of movements (the where pathway).
Major input to Where stream is dominated by M-signals, whereas What stream is dominated by P- and K-signals Kaplan, 2003 V5 Where What Extrastriate visual areas V5 V2 V4
Early evidence for two functional streams from a PET study Dorsal (Parietal) Ventral (Temporal) Haxby et al, 1994 Face & location matching > Control Face Matching > Control Location Matching > Control
Inferior Temporal lobe ( several TE areas) was named as object or image identification areas because IT cells respond well to visual images of natural object (e.g., faces, hands etc.) Temporal lobe
Inferior Temporal (TE) neurons respond well to simplified but critical features of object image Temporal lobe
Inferior temporal neurons are specifically activated by faces. Convergence and integration of key feature information is necessary for identification of a face. From Tanifuji, 2003
Functional MRI activation in human brain Moving stimuli Stationary stimuli V1 (The primary visual cortex) MT/V5 (Extrastriate Area)
Human Functional MRI illustrating Motion Sensitive Areas Tootell et al, 1997 Vandufell et al, 2003 Orban et al, 2003
The functional organization of the visual cortex in primates Dominated by LGN M-cell input Drosal stream for motion perception & spatial localization V5 LIP/7a V2 V4 IT Ventral stream for object recognition & discrimination Dominated by LGN P- & K-cell input
Information processing by neurons of our visual brain 1. Encoding of stimulus contrast and spatial-temporal frequency information (RGC & LGN) 2. Detection and encoding of orientation and spatial frequency of lines and contours (V1) 3. Encoding of angles or curvatures (V2, V4) 4. Encoding the spatial relationships of orientated lines, textures or contours (Linking)(V2, V4) 5. Decoding of linked features Perception of face )(IT, beyond) V5 LIP/7a V2 V4 IT