Searching for simple models
|
|
- Lydia Mosley
- 6 years ago
- Views:
Transcription
1 Searching for simple models 9th Annual Pinkel Endowed Lecture Institute for Research in Cognitive Science University of Pennsylvania Friday April 7 William Bialek Joseph Henry Laboratories of Physics, and Lewis-Sigler Institute for Integrative Genomics Princeton University
2 When we think about (and act upon) the things that we see (and hear, and...), we put them into categories all images from Design Within Reach (
3 g stools office chairs dining chairs benches lounge chairs
4 In the dark of night, vision is based on signals (only) from the rod photoreceptor cells !.5!!.5! !.5!!.5! consider the responses to dim flashes of light.5.5.5!.5!!.5! !.5!.5.5!.5!.5!.5! !.5!!.5! !.5.5! !.5!!.5.5.5!.5!!.5.5!.5! !.5!!.5! !.5!!.5! !.5! !.5!!.5! !.5!.5.5!.5! !.5! !.5!!.5! !.5!!.5! !.5!!.5! ~ microns salamander rods (not that it really matters) rod image and current data from FM Rieke salamander image from MJ Berry II The brain has the problem of categorizing these responses!
5 Remember Hecht, Shlaer & Pirenne! Energy, quanta and vision, J Gen Physiol 5, 89 (94) probability of seeing Hecht Shlaer Pirenne K=6 (inferred) mean number of photons at the retina x-axis is proportional to light intensity of stimulus flash solid line is model where observer sees when more than K photons are counted at the retina... distribution of counts determined by physics of the light source in this regime, our visual perception is controlled by the random arrival of individual photons at the retina categories of rod cell response should correspond to zero, one, two... photons try to categorize based simply on current at peak time probability density (/pa) photons? photon!! current at t peak (pa) photons this gives the right idea, but simplest approach leaves substantial ambiguities (?) is there a better strategy??
6 Raw data for categorization is the current I(t) many time points = many dimensions better categories = better boundaries in this multi-dimensional space predicted bipolar response measured bipolar response try planar boundaries: decisions with output of a linear filter best filter determined by signal and noise properties of the rods themselves normalized voltage.4 photons measured rod (voltage) response. probability density (/pa) optimal filter resolves almost all ambiguity photon!! photons..5. time after light flash (seconds) if there is a unique optimal filtering strategy for processing the rod cell signals, the retina should use this strategy... this is a parameter free prediction! Optimal filtering in the salamander retina. F Rieke, WG Owen & W Bialek, in Advances in Neural Information Processing, R Lippman, J Moody & D Touretzky, eds, pp 77-8 (Morgan Kaufmann, San Mateo CA, 99).
7 categorizing rod responses might be analogous to categorizing images of chairs but sometimes simpler animals actually have to solve the same problem that we do not as different as Mr Larson thinks
8 place a small wire in the back of the fly s head to listen in on the electrical signals from nerve cells that respond to movement The fly has to solve (at least) two problems:! estimate motion from the movie on the retina, and represent or encode the result in the sequence of spikes Optimization principles (as with optimal filtering above): estimates as accurate as possible coding in spikes should be matched to input signals Spikes: Exploring the Neural code F Rieke, D Warland, RR de Ruyter van Steveninck & WB (MIT Press, 997) focus will be on extracting a feature rather than building its representation, i.e., estimation theory not information theory (maybe a mistake for this talk)
9 does the fly make accurate estimates of motion? we can get at this by decoding the spikes from the motion sensitive neurons then look at the power spectrum of errors in the reconstructed signal F ( τ) S(ω) N (ω) N min (ω) v est (t) = i F (t t i ) Reading a neural code WB, F Rieke, RR de Ruyter van Steveninck & D Warland Science 5, 854 (99) compare with the minimum level of errors set by diffraction blur and photoreceptor noise (includes averaging over ~ receptors!) fly approaches optimal estimation on short time scales (high frequencies) that actually matter for behavior
10 What computation must the fly do in order to achieve optimal motion estimates? motion estimation takes photoreceptor signals as inputs (not velocity!) after several layers of processing... output is an estimate of velocity we can actually solve the optimal estimation problem in some limiting cases (also need hypotheses about the statistical structure of the world) Statistical mechanics and visual signal processing M Potters & WB, J Phys I France 4, 755 (994) at high signal-to-noise ratios, velocity is just the ratio of temporal and spatial derivatives v est (t) i (dv i/dt) (V i V i+ ) constant + i (V i V i+ ) V/ t V/ x at low signal-to-noise ratios, the only reliable velocity signal is spatiotemporal correlation v est (t) ij dτ dτ V i (t τ)k ij (τ, τ )V j (t τ ) + optimal estimation always is a tradeoff between systematic and random errors (think about averaging over time in the lab!) the optimal estimator is not perfect... and can even see motion when nothing moves visual stimuli from RR de Ruyter van Steveninck (flies see it move too!) can go further with random stimuli to dissect the computation Features and dimensions: Motion estimation in fly vision WB & RR de Ruyter van Steveninck, (5)
11 Almost everything interesting that the brain does involves LOTS of neurons How do we think about these networks as a whole? Imagine slicing time into little windows, like the frames of a movie. In each frame, each cell either spikes, or it doesn t. a moment ago now a moment later neuron # spike no spike no spike neuron # no spike spike no spike neuron # spike no spike spike neuron # 4 no spike no spike spike New experimental methods make it possible to listen in on many neurons at once (MJ Berry II). neuron #5 no spike spike no spike neuron # 6 no spike no spike spike neuron # 7 no spike spike no spike neuron # 8 spike no spike no spike neuron # 9 no spike no spike no spike neuron # no spike spike spike states of the network: these cells from a salamander retina (not crucial, but cute) These are the words with which the retina tells the brain what we see! How big is the vocabulary? cells = 4 possible words cells =,67,65,6,8,9,45,44,4,7,84 possible words
12 An important insight from theory: If it really is as complicated as it possibly could be, you ll never understand it. Progress = Simplification Simplifying hypothesis #: cells cooperate, but only talk to each other by... collective actions emerge from all pairwise discussions. Simplifying hypothesis #: every cell does its thing independently of all the others. works surprisingly well if you look just at pairs of cells # of cell pairs % correlation independent model experiments! probability fails dramatically if you look at 4 cells (or in detail at cells) / /, one in a million (C Boutin & J Jameson, Princeton Weekly Bulletin, May, 6) 5 5 # of cells that spike together Simplifying hypothesis #: if cells spike together, there must be something special about those cells... (actually, not a simplification). Take seriously the weak correlations among many pairs (compare w/cortex!). Build the least structured model consistent with these correlations. (least structured = maximum entropy) Weak pairwise correlations imply strongly correlated network states in a neural population E Schneidman, MJ Berry II, R Segev & WB, Nature 44, 7 (6).
13 The model we are looking for (minimal structure to match the pairwise correlations) is exactly the Ising model in statistical mechanics. σ i = + neuron i fires a spike σ i = neuron i is silent state of entire network: σ, σ,, σ N {σ i } distribution of network states (words) P (σ, σ,, σ N ) = Z exp h i σ i + J ij σ i σ j i i j for N= we can check the whole distribution! entropy if N neurons are independent Entropy scale Max ent model captures ~9% of the structure max ent given pair correlations actual entropy independent neurons (suggested by weak correlations) pairwise (Ising) model But this is also the Hopfield model of memory! Are there stored patterns?
14 Moving to larger networks opens a much richer structure! with N=4 neurons we have multiple ground states (=stored patterns) network returns to the same basin of attraction when we play the movie again, even if microstate is different observed groups of cells are typical of ensembles generated by drawing means and correlations at random from observed distribution... suggests extrapolation to larger N model from real data can be thought of as being at temperature T= study specific heat vs T (integrate to get entropy) is the system poised near a critical point? (test via adaptation expt!) Ising models for networks of real neurons G Tkacik, E Schneidman, MJ Berry II & WB, (6)
15 How seriously should you take these maximum entropy models? Can we use them to describe more complex phenomena? Try words as networks of letters! one year of Reuters articles (~9 million words) choose 5 most common words select the subset of four letter words (66 distinct words) 4 full probability distribution has (6) ~ half a million elements max ent model consistent with pairwise correlations has ~ 6x(6) ~ 4 parameters, x fewer (!) recall that spelling rules have a very combinatorial feel... if all letters used equally, entropy = 4xlog (6) = 8.8 bits taking account of letter frequencies, entropy of independent letters = 4.59 bits entropy of actual distribution = 7.4 bits so, multi-information = 7.7 bits max-ent model captures 6. bits, or ~87% of the structure inevitably, the model assigns nonzero probability to words not seen in the data... (remember the data are limited to 5 most common words) rite, hove, rase, lost, hive, mave, wark, whet, lise, tame, leat, fave, tike, pall, meek, nate, mast, hale, sime, gave, tome,... Toward a statistical mechanics of four letter words, GJ Stephens & WB, in progress (6).
16 What problem is the brain solving? classification (e.g., rod responses) estimating a feature (e.g., motion)... in many (simple) cases, there is evidence for near-optimal performance (many examples I didn t discuss) if we take this seriously, we have a theory of what the brain should compute; key qualitative prediction is context dependence but: why these features? (laundry list problem) is there a unifying theme for the problems that the brain solves well? are all the problems really the problem of prediction? How do neurons cooperate in networks? common observation is that pairs of neurons are only weakly correlated or anti-correlated but there are LOTS of pairs from (simple) statistical mechanics models: if all pairs interact, weak is <</N, not << minimally structured models consistent with weak correlations predict dramatic collective states: memories, critical points... (exotica implied by modest phenomena!) how do we connect network dynamics and computational function? maybe: predictive information is maximal at critical points...
encoding and estimation bottleneck and limits to visual fidelity
Retina Light Optic Nerve photoreceptors encoding and estimation bottleneck and limits to visual fidelity interneurons ganglion cells light The Neural Coding Problem s(t) {t i } Central goals for today:
More informationOptimal Filtering in the Salamander Retina
Optimal Filtering in the Salamander Retina Fred Riekea,c, W. Geoffrey Owenb and William Bialeka,b,c Departments of Physicsa and Molecular and Cell Biologyb University of California Berkeley, California
More informationInformation Theory. Mark van Rossum. January 24, School of Informatics, University of Edinburgh 1 / 35
1 / 35 Information Theory Mark van Rossum School of Informatics, University of Edinburgh January 24, 2018 0 Version: January 24, 2018 Why information theory 2 / 35 Understanding the neural code. Encoding
More informationDo Neurons Process Information Efficiently?
Do Neurons Process Information Efficiently? James V Stone, University of Sheffield Claude Shannon, 1916-2001 Nothing in biology makes sense except in the light of evolution. Theodosius Dobzhansky, 1973.
More informationThe homogeneous Poisson process
The homogeneous Poisson process during very short time interval Δt there is a fixed probability of an event (spike) occurring independent of what happened previously if r is the rate of the Poisson process,
More informationMembrane equation. VCl. dv dt + V = V Na G Na + V K G K + V Cl G Cl. G total. C m. G total = G Na + G K + G Cl
Spiking neurons Membrane equation V GNa GK GCl Cm VNa VK VCl dv dt + V = V Na G Na + V K G K + V Cl G Cl G total G total = G Na + G K + G Cl = C m G total Membrane with synaptic inputs V Gleak GNa GK
More informationTransformation of stimulus correlations by the retina
Transformation of stimulus correlations by the retina Kristina Simmons (University of Pennsylvania) and Jason Prentice, (now Princeton University) with Gasper Tkacik (IST Austria) Jan Homann (now Princeton
More informationAnalyzing large-scale spike trains data with spatio-temporal constraints
Author manuscript, published in "NeuroComp/KEOpS'12 workshop beyond the retina: from computational models to outcomes in bioengineering. Focus on architecture and dynamics sustaining information flows
More informationBrains and Computation
15-883: Computational Models of Neural Systems Lecture 1.1: Brains and Computation David S. Touretzky Computer Science Department Carnegie Mellon University 1 Models of the Nervous System Hydraulic network
More informationarxiv:physics/ v1 [physics.data-an] 7 Jun 2003
Entropy and information in neural spike trains: Progress on the sampling problem arxiv:physics/0306063v1 [physics.data-an] 7 Jun 2003 Ilya Nemenman, 1 William Bialek, 2 and Rob de Ruyter van Steveninck
More informationBASIC VISUAL SCIENCE CORE
BASIC VISUAL SCIENCE CORE Absolute and Increment Thresholds Ronald S. Harwerth Fall, 2016 1. Psychophysics of Vision 2. Light and Dark Adaptation Michael Kalloniatis and Charles Luu 1 The Neuron Doctrine
More information4.2 Entropy lost and information gained
4.2. ENTROPY LOST AND INFORMATION GAINED 101 4.2 Entropy lost and information gained Returning to the conversation between Max and Allan, we assumed that Max would receive a complete answer to his question,
More informationarxiv:q-bio/ v1 [q-bio.nc] 2 May 2005
Features and dimensions: Motion estimation in fly vision William Bialek a and Rob R. de Ruyter van Steveninck b a Joseph Henry Laboratories of Physics, b Department of Molecular Biology, and the Lewis
More informationAnalyzing large-scale spike trains data with spatio-temporal constraints
Analyzing large-scale spike trains data with spatio-temporal constraints Hassan Nasser, Olivier Marre, Selim Kraria, Thierry Viéville, Bruno Cessac To cite this version: Hassan Nasser, Olivier Marre, Selim
More informationSpatial Vision: Primary Visual Cortex (Chapter 3, part 1)
Spatial Vision: Primary Visual Cortex (Chapter 3, part 1) Lecture 6 Jonathan Pillow Sensation & Perception (PSY 345 / NEU 325) Princeton University, Spring 2015 1 Chapter 2 remnants 2 Receptive field:
More informationAdaptation in the Neural Code of the Retina
Adaptation in the Neural Code of the Retina Lens Retina Fovea Optic Nerve Optic Nerve Bottleneck Neurons Information Receptors: 108 95% Optic Nerve 106 5% After Polyak 1941 Visual Cortex ~1010 Mean Intensity
More informationThe Bayesian Brain. Robert Jacobs Department of Brain & Cognitive Sciences University of Rochester. May 11, 2017
The Bayesian Brain Robert Jacobs Department of Brain & Cognitive Sciences University of Rochester May 11, 2017 Bayesian Brain How do neurons represent the states of the world? How do neurons represent
More information+ + ( + ) = Linear recurrent networks. Simpler, much more amenable to analytic treatment E.g. by choosing
Linear recurrent networks Simpler, much more amenable to analytic treatment E.g. by choosing + ( + ) = Firing rates can be negative Approximates dynamics around fixed point Approximation often reasonable
More informationFeatures and dimensions: Motion estimation in fly vision
Features and dimensions: Motion estimation in fly vision William Bialek a and Rob R. de Ruyter van Steveninck b a Joseph Henry Laboratories of Physics, and Lewis Sigler Institute for Integrative Genomics
More informationBayesian probability theory and generative models
Bayesian probability theory and generative models Bruno A. Olshausen November 8, 2006 Abstract Bayesian probability theory provides a mathematical framework for peforming inference, or reasoning, using
More informationarxiv:cond-mat/ v2 27 Jun 1997
Entropy and Information in Neural Spike rains Steven P. Strong, 1 Roland Koberle, 1,2 Rob R. de Ruyter van Steveninck, 1 and William Bialek 1 1 NEC Research Institute, 4 Independence Way, Princeton, New
More informationEstimation of information-theoretic quantities
Estimation of information-theoretic quantities Liam Paninski Gatsby Computational Neuroscience Unit University College London http://www.gatsby.ucl.ac.uk/ liam liam@gatsby.ucl.ac.uk November 16, 2004 Some
More informationLearning the collective dynamics of complex biological systems. from neurons to animal groups. Thierry Mora
Learning the collective dynamics of complex biological systems from neurons to animal groups Thierry Mora Università Sapienza Rome A. Cavagna I. Giardina O. Pohl E. Silvestri M. Viale Aberdeen University
More informationLimulus. The Neural Code. Response of Visual Neurons 9/21/2011
Crab cam (Barlow et al., 2001) self inhibition recurrent inhibition lateral inhibition - L16. Neural processing in Linear Systems: Temporal and Spatial Filtering C. D. Hopkins Sept. 21, 2011 The Neural
More informationRESEARCH STATEMENT. Nora Youngs, University of Nebraska - Lincoln
RESEARCH STATEMENT Nora Youngs, University of Nebraska - Lincoln 1. Introduction Understanding how the brain encodes information is a major part of neuroscience research. In the field of neural coding,
More information1/12/2017. Computational neuroscience. Neurotechnology.
Computational neuroscience Neurotechnology https://devblogs.nvidia.com/parallelforall/deep-learning-nutshell-core-concepts/ 1 Neurotechnology http://www.lce.hut.fi/research/cogntech/neurophysiology Recording
More informationHopfield Neural Network and Associative Memory. Typical Myelinated Vertebrate Motoneuron (Wikipedia) Topic 3 Polymers and Neurons Lecture 5
Hopfield Neural Network and Associative Memory Typical Myelinated Vertebrate Motoneuron (Wikipedia) PHY 411-506 Computational Physics 2 1 Wednesday, March 5 1906 Nobel Prize in Physiology or Medicine.
More informationIntroduction to Algorithms / Algorithms I Lecturer: Michael Dinitz Topic: Intro to Learning Theory Date: 12/8/16
600.463 Introduction to Algorithms / Algorithms I Lecturer: Michael Dinitz Topic: Intro to Learning Theory Date: 12/8/16 25.1 Introduction Today we re going to talk about machine learning, but from an
More informationSolving with Absolute Value
Solving with Absolute Value Who knew two little lines could cause so much trouble? Ask someone to solve the equation 3x 2 = 7 and they ll say No problem! Add just two little lines, and ask them to solve
More informationEfficient representation as a design principle for neural coding and computation
Efficient representation as a design principle for neural coding and computation William Bialek, a Rob R. de Ruyter van Steveninck b and Naftali Tishby c a Joseph Henry Laboratories of Physics, Lewis Sigler
More informationDo students sleep the recommended 8 hours a night on average?
BIEB100. Professor Rifkin. Notes on Section 2.2, lecture of 27 January 2014. Do students sleep the recommended 8 hours a night on average? We first set up our null and alternative hypotheses: H0: μ= 8
More informationIn other words, we are interested in what is happening to the y values as we get really large x values and as we get really small x values.
Polynomial functions: End behavior Solutions NAME: In this lab, we are looking at the end behavior of polynomial graphs, i.e. what is happening to the y values at the (left and right) ends of the graph.
More informationResonance and response
Chapter 2 Resonance and response Last updated September 20, 2008 In this section of the course we begin with a very simple system a mass hanging from a spring and see how some remarkable ideas emerge.
More informationThe Philosophy of Physics. Is Space Absolute or Relational?
The Philosophy of Physics Lecture Two Is Space Absolute or Relational? Rob Trueman rob.trueman@york.ac.uk University of York Newton s Absolute Motion and Acceleration Is Space Absolute or Relational? Newton
More informationSpecial Theory of Relativity Prof. Shiva Prasad Department of Physics Indian Institute of Technology, Bombay. Lecture - 15 Momentum Energy Four Vector
Special Theory of Relativity Prof. Shiva Prasad Department of Physics Indian Institute of Technology, Bombay Lecture - 15 Momentum Energy Four Vector We had started discussing the concept of four vectors.
More informationSPIKE TRIGGERED APPROACHES. Odelia Schwartz Computational Neuroscience Course 2017
SPIKE TRIGGERED APPROACHES Odelia Schwartz Computational Neuroscience Course 2017 LINEAR NONLINEAR MODELS Linear Nonlinear o Often constrain to some form of Linear, Nonlinear computations, e.g. visual
More informationIsing models for neural activity inferred via Selective Cluster Expansion: structural and coding properties
Ising models for neural activity inferred via Selective Cluster Expansion: structural and coding properties John Barton,, and Simona Cocco Department of Physics, Rutgers University, Piscataway, NJ 8854
More informationBayesian Inference. 2 CS295-7 cfl Michael J. Black,
Population Coding Now listen to me closely, young gentlemen. That brain is thinking. Maybe it s thinking about music. Maybe it has a great symphony all thought out or a mathematical formula that would
More informationSeptember 16, 2004 The NEURON Book: Chapter 2
Chapter 2 The ing perspective This and the following chapter deal with concepts that are not NEURON-specific but instead pertain equally well to any tools used for neural ing. Why? In order to achieve
More informationIdentification of Odors by the Spatiotemporal Dynamics of the Olfactory Bulb. Outline
Identification of Odors by the Spatiotemporal Dynamics of the Olfactory Bulb Henry Greenside Department of Physics Duke University Outline Why think about olfaction? Crash course on neurobiology. Some
More informationAdaptive contrast gain control and information maximization $
Neurocomputing 65 66 (2005) 6 www.elsevier.com/locate/neucom Adaptive contrast gain control and information maximization $ Yuguo Yu a,, Tai Sing Lee b a Center for the Neural Basis of Cognition, Carnegie
More informationEffects of Interactive Function Forms in a Self-Organized Critical Model Based on Neural Networks
Commun. Theor. Phys. (Beijing, China) 40 (2003) pp. 607 613 c International Academic Publishers Vol. 40, No. 5, November 15, 2003 Effects of Interactive Function Forms in a Self-Organized Critical Model
More informationTemporal filtering in retinal bipolar cells
Temporal filtering in retinal bipolar cells Elements of an optimal computation? William Bialek*t and W. Geoffrey Owen$ Departments of *Physics and tmolecular and Cell Biology, University of California
More informationLesson 6-1: Relations and Functions
I ll bet you think numbers are pretty boring, don t you? I ll bet you think numbers have no life. For instance, numbers don t have relationships do they? And if you had no relationships, life would be
More informationSUPPLEMENTARY INFORMATION
Spatio-temporal correlations and visual signaling in a complete neuronal population Jonathan W. Pillow 1, Jonathon Shlens 2, Liam Paninski 3, Alexander Sher 4, Alan M. Litke 4,E.J.Chichilnisky 2, Eero
More informationCSC321 Lecture 16: ResNets and Attention
CSC321 Lecture 16: ResNets and Attention Roger Grosse Roger Grosse CSC321 Lecture 16: ResNets and Attention 1 / 24 Overview Two topics for today: Topic 1: Deep Residual Networks (ResNets) This is the state-of-the
More informationCS1800: Mathematical Induction. Professor Kevin Gold
CS1800: Mathematical Induction Professor Kevin Gold Induction: Used to Prove Patterns Just Keep Going For an algorithm, we may want to prove that it just keeps working, no matter how big the input size
More informationMid Year Project Report: Statistical models of visual neurons
Mid Year Project Report: Statistical models of visual neurons Anna Sotnikova asotniko@math.umd.edu Project Advisor: Prof. Daniel A. Butts dab@umd.edu Department of Biology Abstract Studying visual neurons
More informationMachine Learning. Neural Networks
Machine Learning Neural Networks Bryan Pardo, Northwestern University, Machine Learning EECS 349 Fall 2007 Biological Analogy Bryan Pardo, Northwestern University, Machine Learning EECS 349 Fall 2007 THE
More informationMarkov Chain Monte Carlo The Metropolis-Hastings Algorithm
Markov Chain Monte Carlo The Metropolis-Hastings Algorithm Anthony Trubiano April 11th, 2018 1 Introduction Markov Chain Monte Carlo (MCMC) methods are a class of algorithms for sampling from a probability
More informationNeural Coding: Integrate-and-Fire Models of Single and Multi-Neuron Responses
Neural Coding: Integrate-and-Fire Models of Single and Multi-Neuron Responses Jonathan Pillow HHMI and NYU http://www.cns.nyu.edu/~pillow Oct 5, Course lecture: Computational Modeling of Neuronal Systems
More information7 Rate-Based Recurrent Networks of Threshold Neurons: Basis for Associative Memory
Physics 178/278 - David Kleinfeld - Fall 2005; Revised for Winter 2017 7 Rate-Based Recurrent etworks of Threshold eurons: Basis for Associative Memory 7.1 A recurrent network with threshold elements The
More information213 Midterm coming up
213 Midterm coming up Monday April 8 @ 7 pm (conflict exam @ 5:15pm) Covers: Lectures 1-12 (not including thermal radiation) HW 1-4 Discussion 1-4 Labs 1-2 Review Session Sunday April 7, 3-5 PM, 141 Loomis
More informationCapacitors. Chapter How capacitors work Inside a capacitor
Chapter 6 Capacitors In every device we have studied so far sources, resistors, diodes and transistors the relationship between voltage and current depends only on the present, independent of the past.
More informationCOMP 546. Lecture 21. Cochlea to brain, Source Localization. Tues. April 3, 2018
COMP 546 Lecture 21 Cochlea to brain, Source Localization Tues. April 3, 2018 1 Ear pinna auditory canal cochlea outer middle inner 2 Eye Ear Lens? Retina? Photoreceptors (light -> chemical) Ganglion cells
More informationFisher Information Quantifies Task-Specific Performance in the Blowfly Photoreceptor
Fisher Information Quantifies Task-Specific Performance in the Blowfly Photoreceptor Peng Xu and Pamela Abshire Department of Electrical and Computer Engineering and the Institute for Systems Research
More informationCHAPTER 3. Pattern Association. Neural Networks
CHAPTER 3 Pattern Association Neural Networks Pattern Association learning is the process of forming associations between related patterns. The patterns we associate together may be of the same type or
More informationLearning Outcomes 2. Key Concepts 2. Misconceptions and Teaching Challenges 3. Vocabulary 4. Lesson and Content Overview 5
UNIT 1 GUIDE Table of Contents Learning Outcomes 2 Key Concepts 2 Misconceptions and Teaching Challenges 3 Vocabulary 4 Lesson and Content Overview 5 BIG HISTORY PROJECT / UNIT 1 GUIDE 1 Unit 1 What Is
More informationMultiple Testing. Gary W. Oehlert. January 28, School of Statistics University of Minnesota
Multiple Testing Gary W. Oehlert School of Statistics University of Minnesota January 28, 2016 Background Suppose that you had a 20-sided die. Nineteen of the sides are labeled 0 and one of the sides is
More informationArtificial Neural Networks. Q550: Models in Cognitive Science Lecture 5
Artificial Neural Networks Q550: Models in Cognitive Science Lecture 5 "Intelligence is 10 million rules." --Doug Lenat The human brain has about 100 billion neurons. With an estimated average of one thousand
More informationPhysics 403. Segev BenZvi. Credible Intervals, Confidence Intervals, and Limits. Department of Physics and Astronomy University of Rochester
Physics 403 Credible Intervals, Confidence Intervals, and Limits Segev BenZvi Department of Physics and Astronomy University of Rochester Table of Contents 1 Summarizing Parameters with a Range Bayesian
More information1 Probabilities. 1.1 Basics 1 PROBABILITIES
1 PROBABILITIES 1 Probabilities Probability is a tricky word usually meaning the likelyhood of something occuring or how frequent something is. Obviously, if something happens frequently, then its probability
More informationOnce Upon A Time, There Was A Certain Ludwig
Once Upon A Time, There Was A Certain Ludwig Statistical Mechanics: Ensembles, Distributions, Entropy and Thermostatting Srinivas Mushnoori Chemical & Biochemical Engineering Rutgers, The State University
More informationClassification and Regression Trees
Classification and Regression Trees Ryan P Adams So far, we have primarily examined linear classifiers and regressors, and considered several different ways to train them When we ve found the linearity
More informationReplay argument. Abstract. Tanasije Gjorgoski Posted on on 03 April 2006
Replay argument Tanasije Gjorgoski Posted on http://broodsphilosophy.wordpress.com/, on 03 April 2006 Abstract Before a year or so ago, I was trying to think of an example so I can properly communicate
More informationLearning the dynamics of biological networks
Learning the dynamics of biological networks Thierry Mora ENS Paris A.Walczak Università Sapienza Rome A. Cavagna I. Giardina O. Pohl E. Silvestri M.Viale Aberdeen University F. Ginelli Laboratoire de
More information7 Recurrent Networks of Threshold (Binary) Neurons: Basis for Associative Memory
Physics 178/278 - David Kleinfeld - Winter 2019 7 Recurrent etworks of Threshold (Binary) eurons: Basis for Associative Memory 7.1 The network The basic challenge in associative networks, also referred
More informationMathematical Tools for Neuroscience (NEU 314) Princeton University, Spring 2016 Jonathan Pillow. Homework 8: Logistic Regression & Information Theory
Mathematical Tools for Neuroscience (NEU 34) Princeton University, Spring 206 Jonathan Pillow Homework 8: Logistic Regression & Information Theory Due: Tuesday, April 26, 9:59am Optimization Toolbox One
More informationChapter 9: The Perceptron
Chapter 9: The Perceptron 9.1 INTRODUCTION At this point in the book, we have completed all of the exercises that we are going to do with the James program. These exercises have shown that distributed
More informationCPSC 340: Machine Learning and Data Mining. MLE and MAP Fall 2017
CPSC 340: Machine Learning and Data Mining MLE and MAP Fall 2017 Assignment 3: Admin 1 late day to hand in tonight, 2 late days for Wednesday. Assignment 4: Due Friday of next week. Last Time: Multi-Class
More informationarxiv:physics/ v2 [physics.bio-ph] 9 Jul 2002
arxiv:physics/0205030v2 [physics.bio-ph] 9 Jul 2002 Thinking about the brain Based on lectures at Les Houches Session LXXV, July 2001 William Bialek NEC Research Institute, 4 Independence Way, Princeton,
More informationLecture 11: Extrema. Nathan Pflueger. 2 October 2013
Lecture 11: Extrema Nathan Pflueger 2 October 201 1 Introduction In this lecture we begin to consider the notion of extrema of functions on chosen intervals. This discussion will continue in the lectures
More information37-6 Watching the electrons (matter waves)
37-6 Watching the electrons (matter waves) 1 testing our proposition: the electrons go either through hole 1 or hole 2 add a very strong light source behind walls between two holes, electrons will scatter
More informationComputation in a single neuron: Hodgkin and Huxley revisited
Computation in a single neuron: Hodgkin and Huxley revisited Blaise Agüera y Arcas, 1 Adrienne L. Fairhall, 2,3 and William Bialek 2,4 1 Rare Books Library, Princeton University, Princeton, New Jersey
More informationNeural coding Ecological approach to sensory coding: efficient adaptation to the natural environment
Neural coding Ecological approach to sensory coding: efficient adaptation to the natural environment Jean-Pierre Nadal CNRS & EHESS Laboratoire de Physique Statistique (LPS, UMR 8550 CNRS - ENS UPMC Univ.
More informationMITOCW ocw f99-lec01_300k
MITOCW ocw-18.06-f99-lec01_300k Hi. This is the first lecture in MIT's course 18.06, linear algebra, and I'm Gilbert Strang. The text for the course is this book, Introduction to Linear Algebra. And the
More informationNonlinear reverse-correlation with synthesized naturalistic noise
Cognitive Science Online, Vol1, pp1 7, 2003 http://cogsci-onlineucsdedu Nonlinear reverse-correlation with synthesized naturalistic noise Hsin-Hao Yu Department of Cognitive Science University of California
More informationProbability and Independence Terri Bittner, Ph.D.
Probability and Independence Terri Bittner, Ph.D. The concept of independence is often confusing for students. This brief paper will cover the basics, and will explain the difference between independent
More informationCMPSCI611: Three Divide-and-Conquer Examples Lecture 2
CMPSCI611: Three Divide-and-Conquer Examples Lecture 2 Last lecture we presented and analyzed Mergesort, a simple divide-and-conquer algorithm. We then stated and proved the Master Theorem, which gives
More informationConcerns of the Psychophysicist. Three methods for measuring perception. Yes/no method of constant stimuli. Detection / discrimination.
Three methods for measuring perception Concerns of the Psychophysicist. Magnitude estimation 2. Matching 3. Detection/discrimination Bias/ Attentiveness Strategy/Artifactual Cues History of stimulation
More informationAN ALGEBRA PRIMER WITH A VIEW TOWARD CURVES OVER FINITE FIELDS
AN ALGEBRA PRIMER WITH A VIEW TOWARD CURVES OVER FINITE FIELDS The integers are the set 1. Groups, Rings, and Fields: Basic Examples Z := {..., 3, 2, 1, 0, 1, 2, 3,...}, and we can add, subtract, and multiply
More informationIntroduction to MRI Acquisition
Introduction to MRI Acquisition James Meakin FMRIB Physics Group FSL Course, Bristol, September 2012 1 What are we trying to achieve? 2 What are we trying to achieve? Informed decision making: Protocols
More informationNatural Image Statistics and Neural Representations
Natural Image Statistics and Neural Representations Michael Lewicki Center for the Neural Basis of Cognition & Department of Computer Science Carnegie Mellon University? 1 Outline 1. Information theory
More informationChapter 2. Mathematical Reasoning. 2.1 Mathematical Models
Contents Mathematical Reasoning 3.1 Mathematical Models........................... 3. Mathematical Proof............................ 4..1 Structure of Proofs........................ 4.. Direct Method..........................
More informationConvolution and Linear Systems
CS 450: Introduction to Digital Signal and Image Processing Bryan Morse BYU Computer Science Introduction Analyzing Systems Goal: analyze a device that turns one signal into another. Notation: f (t) g(t)
More informationNeuroscience Introduction
Neuroscience Introduction The brain As humans, we can identify galaxies light years away, we can study particles smaller than an atom. But we still haven t unlocked the mystery of the three pounds of matter
More informationDiscrete Mathematics and Probability Theory Fall 2014 Anant Sahai Note 15. Random Variables: Distributions, Independence, and Expectations
EECS 70 Discrete Mathematics and Probability Theory Fall 204 Anant Sahai Note 5 Random Variables: Distributions, Independence, and Expectations In the last note, we saw how useful it is to have a way of
More information15-451/651: Design & Analysis of Algorithms September 13, 2018 Lecture #6: Streaming Algorithms last changed: August 30, 2018
15-451/651: Design & Analysis of Algorithms September 13, 2018 Lecture #6: Streaming Algorithms last changed: August 30, 2018 Today we ll talk about a topic that is both very old (as far as computer science
More informationAQI: Advanced Quantum Information Lecture 6 (Module 2): Distinguishing Quantum States January 28, 2013
AQI: Advanced Quantum Information Lecture 6 (Module 2): Distinguishing Quantum States January 28, 2013 Lecturer: Dr. Mark Tame Introduction With the emergence of new types of information, in this case
More informationThe World According to Wolfram
The World According to Wolfram Basic Summary of NKS- A New Kind of Science is Stephen Wolfram s attempt to revolutionize the theoretical and methodological underpinnings of the universe. Though this endeavor
More informationPeriod Analysis on a Spreadsheet
Aliases in Depth An alias for a period is another period where the data seemingly fits as well, or nearly so, as the correct period. The most common encounter with aliasing is when you observe a target
More informationSection 20: Arrow Diagrams on the Integers
Section 0: Arrow Diagrams on the Integers Most of the material we have discussed so far concerns the idea and representations of functions. A function is a relationship between a set of inputs (the leave
More informationTHE retina in general consists of three layers: photoreceptors
CS229 MACHINE LEARNING, STANFORD UNIVERSITY, DECEMBER 2016 1 Models of Neuron Coding in Retinal Ganglion Cells and Clustering by Receptive Field Kevin Fegelis, SUID: 005996192, Claire Hebert, SUID: 006122438,
More informationEntropy and information in neural spike trains: Progress on the sampling problem
PHYSICAL REVIEW E 69, 056111 (2004) Entropy and information in neural spike trains: Progress on the sampling problem Ilya Nemenman, 1, * William Bialek, 2, and Rob de Ruyter van Steveninck 3, 1 avli Institute
More informationCPSC 340: Machine Learning and Data Mining
CPSC 340: Machine Learning and Data Mining MLE and MAP Original version of these slides by Mark Schmidt, with modifications by Mike Gelbart. 1 Admin Assignment 4: Due tonight. Assignment 5: Will be released
More information1 Boolean Algebra Simplification
cs281: Computer Organization Lab3 Prelab Our objective in this prelab is to lay the groundwork for simplifying boolean expressions in order to minimize the complexity of the resultant digital logic circuit.
More informationArtificial Neural Networks Examination, June 2005
Artificial Neural Networks Examination, June 2005 Instructions There are SIXTY questions. (The pass mark is 30 out of 60). For each question, please select a maximum of ONE of the given answers (either
More informationMath 138: Introduction to solving systems of equations with matrices. The Concept of Balance for Systems of Equations
Math 138: Introduction to solving systems of equations with matrices. Pedagogy focus: Concept of equation balance, integer arithmetic, quadratic equations. The Concept of Balance for Systems of Equations
More informationModeling Convergent ON and OFF Pathways in the Early Visual System
Modeling Convergent ON and OFF Pathways in the Early Visual System The Harvard community has made this article openly available. Please share how this access benefits you. Your story matters Citation Gollisch,
More informationPlease bring the task to your first physics lesson and hand it to the teacher.
Pre-enrolment task for 2014 entry Physics Why do I need to complete a pre-enrolment task? This bridging pack serves a number of purposes. It gives you practice in some of the important skills you will
More information