Probing Real Sensory Worlds of Receivers with Unsupervised Clustering

Transcription

1 with Unsupervised Clustering Michael Pfeiffer 1,2 *, Manfred Hartbauer 3, Alexander B. Lang 3, Wolfgang Maass 1, Heinrich Römer 3 1 Institute for Theoretical Computer Science, TU Graz, Graz, Austria, 2 Institute of Neuroinformatics, University of Zurich and ETH Zurich, Zurich, Switzerland, 3 Institute for Zoology, Karl Franzens University, Graz, Austria Abstract The task of an organism to extract information about the external environment from sensory signals is based entirely on the analysis of ongoing afferent spike activity provided by the sense organs. We investigate the processing of auditory stimuli by an acoustic interneuron of insects. In contrast to most previous work we do this by using stimuli and neurophysiological recordings directly in the nocturnal tropical rainforest, where the insect communicates. Different from typical recordings in sound proof laboratories, strong environmental noise from multiple sound sources interferes with the perception of acoustic signals in these realistic scenarios. We apply a recently developed unsupervised machine learning algorithm based on probabilistic inference to find frequently occurring firing patterns in the response of the acoustic interneuron. We can thus ask how much information the central nervous system of the receiver can extract from bursts without ever being told which type and which variants of bursts are characteristic for particular stimuli. Our results show that the reliability of burst coding in the time domain is so high that identical stimuli lead to extremely similar spike pattern responses, even for different preparations on different dates, and even if one of the preparations is recorded outdoors and the other one in the sound proof lab. Simultaneous recordings in two preparations exposed to the same acoustic environment reveal that characteristics of burst patterns are largely preserved among individuals of the same species. Our study shows that burst coding can provide a reliable mechanism for acoustic insects to classify and discriminate signals under very noisy real-world conditions. This gives new insights into the neural mechanisms potentially used by bushcrickets to discriminate conspecific songs from sounds of predators in similar carrier frequency bands. Citation: Pfeiffer M, Hartbauer M, Lang AB, Maass W, Römer H (2012) Probing Real Sensory Worlds of Receivers with Unsupervised Clustering. PLoS ONE 7(6): e doi: /journal.pone Editor: Daniel Durstewitz, Heidelberg University, Germany Received October 20, 2011; Accepted April 19, 2012; Published June 6, 2012 Copyright: ß 2012 Pfeiffer et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Funding: This project was supported by the Austrian Science Fund (FWF P20882-B09) to HR, the Austrian Academy for Sciences and the Karl-Franzens-University of Graz to ABL. MP has been supported by a Forschungskredit grant of the University of Zurich. Written under partial support by the European Union project # FP (PASCAL2), project #FP (BRAIN-I-NETS), and project # (BrainScaleS) to WM. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Competing Interests: The authors have declared that no competing interests exist. * pfeiffer@ini.phys.ethz.ch Introduction In order to fulfill its task of shaping the behavior of organisms, the sensory system and the brain have to rely on information about the outside physical world, provided by the sense organs, which respond to different forms of energy. The information is transmitted via afferent nerves and encoded in trains of action potentials. The brain, by decoding this information, has to make adaptive assumptions about what had happened in the physical world. A central issue in sensory physiology deals with the coding and decoding mechanism(s) in the sense organs and central nervous system, respectively. Whereas early work concentrated on information provided by the average spike count over an appropriate time window (or firing rate in action potentials/ second), it soon became clear that codes using the precise timing of action potentials would make more efficient use of the capacity of afferent lines to the brain. Yet, the mechanisms by which stimuli are represented in the timing of spikes are still not fully understood [1 3]. Irrespective of the sensory system investigated, recordings of single sensory neurons, or first-order sensory interneurons, always reveal isolated spikes and spikes grouped as bursts, i.e. short episodes of high-frequency action potential firing (e.g. [4 6]). These bursts - in contrast to single spikes-have been suggested to have particular importance for the function of the brain (review [7]), and in sensory systems bursts convey the important stimulus features [6,8]. Yet, the problem of extraction of characteristic features within these bursts for identifying stimulus features and for object classification is difficult because spike trains exhibit variability. For insects and the acoustic modality, [9] reviewed the sources for such variability, and how it affects the processing of temporal patterns of acoustic signals. For example, one important source for such variability in the auditory modality results from the fact that in real world situations individuals are exposed to multiple sound sources, originating from different locations, or that signals are degraded and attenuated on the transmission channel between sender and receiver [10 14]. Internal noise as a result of stochastic processes within the nervous system is a further source for spike train variability. As a result of the unavoidable noisiness of spike trains in neurons of sensory pathways one should expect the evolution of mechanisms in the nervous system leading to a reduction of the effects caused by false stimulus feature extraction and/or classification due to noise. On the other hand, minute variations PLoS ONE 1 June 2012 Volume 7 Issue 6 e37354

2 in spike trains may well reflect differences between objects or object classes which are important for the receiver, such as small differences in the size of a sender, or the loudness or frequency composition in the sound signal of a mate. Such small differences, in contrast to those caused by noise, should be preserved during sensory processing, since they represent the neuronal basis for discrimination between mates or other decisions of importance for the receiver. If bursts of action potentials contain the information about relevant features of objects or object classes, it should be possible to unambiguously distinguish 1) bursts of spikes elicited in response to a given stimulus from those bursts which resulted from noise, and 2) from bursts elicited in response to stimuli with different features. Various attempts have been made in the past to identify algorithms for such a task. In this paper we present a set of machine learning tools to analyze and discriminate burst data while preserving most of the information about precise firing times, which is crucial within the auditory system. Our approach combines the Victor-Purpura spike metric [15 17] and the recently developed affinity propagation algorithm [18], a non-parametric clustering algorithm based on principles from probabilistic inference. Affinity propagation has lead to excellent results for a number of large datasets, and our study presents its first application to the discovery of burst patterns. This allows us to find meaningful spike patterns also in the responses to environmental noise signals, which may carry information about the identity or location of different sound sources. We present data from a model system using an identified neuron approach in an acoustically communicating insect. This system offers several advantages for studying sensory burst coding over previous ones: 1) All recordings stem from the same identified neuron (called omega-neuron; [19]) in different preparations. 2) The first-order neuron in the auditory pathway integrates sensory information from a very limited number of receptor cells in the ear (20 40 receptors). 3) Recordings can be obtained for several hours, and most importantly, 4) portable preparations have been developed to make recordings directly in the insects natural environment, such as the tropical rainforest [13,20,21]. This permits to study sensory coding under the most natural conditions possible. Broadcasting well defined acoustic stimuli from some distance to the preparation, while recording the response of the neuron to these stimuli and to the background noise allows us to gain new insights about the characteristics and reliability of burst coding. Results The Omega Neuron and Experimental Setup A total of 27 adult male and female bushcrickets (Docidocercus gigliotosi) were used for this study. We recorded the activity of an identified auditory interneuron, the so-called omega neuron, in the field, using a technique introduced in [20] and [22], and explained in more detail in the Materials and Methods section. The morphology of the cell, as revealed from intracellular dye injection, is shown in Figure 1A (inset). It is a local neuron in the prothoracic ganglion and receives excitatory input from almost all of the receptors in the hearing organ [23]. The tuning of the cell reflects the broad-band hearing sensitivity of the insect, matching both the frequencies of the conspecific calling song, and ultrasonic frequencies up to 100 khz, thus including bat echolocation calls as well. As in other bushcricket species, the sensitivity of auditory receptors in D. gigliotosi differs by only a few db from the sensitivity of the omega cell at most frequencies except below 5 khz [24]. Furthermore, in response to a stimulus above its threshold, the neuron fires bursts of action potentials and copies the temporal pattern of an acoustic stimulus in a tonic manner. Altogether these attributes make outdoor recordings of the activity of the omega cell very suitable for studying sensory coding under realistic, i.e. outdoor conditions in the animals own natural habitat. The study was conducted between 2002 and 2004 on Barro Colorado Island, located in central Panama within Gatún Lake, part of the Panama Canal. D. gigliotosi is a tropical insect living predominantly in the rainforest understorey, and all its activity, including acoustic communication, is restricted to the night. Thus, all recordings were made during times after sunset (about 6 p.m. local time) except for control measurements (see Materials and Methods). During some of the recording sessions, five different stimuli at an intensity 20 db above the threshold of the preparation were broadcast through a speaker. The stimuli differed in duration and the number of pulses (see Figure 1B), and were broadcasted every 10 seconds, which is within the range of the naturally occurring intervals in the calling song of this insect [21]. Males produce single or double syllables, repeated typically once every 10 seconds. Thus, the stimulus classes 1,2, and 3 in Figure 1B can be seen as representative for the variation of conspecific signals. Classes 4 and 5 are artificial stimuli that were used for control, and never occur in this species. Burst Coding of Acoustic Signals in Natural Habitats A typical result for the effect of background noise on sensory coding is shown in Figure 1. The receiver was placed within the rainforest at hrs 2 m from a speaker broadcasting a single sound pulse of 10 ms, at a sound pressure level of 20 db above the threshold of the cell (which corresponds to intermediate distances of 5 10 m between sender and receiver). Since a female has no a priori knowledge about the presence of a male signal, her only information about a signal is encoded in afferent nervous activity such as the one shown in the upper recording. Artificial acoustic signals, as well as some natural background stimuli caused bursting activity in the nerve cell, i.e. it was firing at a much increased rate compared to its baseline firing activity. We extracted from the continuous recordings in natural habitats all these short time segments in which the omega neuron was bursting. Our criterion for detecting bursts in a continuous stream of spikes required a silent interval of at least 60 ms before the start of the burst, a constantly high firing rate of at least 33 Hz, a minimum duration of 8 ms, and a minimum of 5 spikes within the burst (see Figure 2D, as well as Burst Detection in Materials and Methods for more details). In Figure 2 we illustrate the analysis of one recording session. From the joint interspike-interval (ISI) diagram in Figure 2A, which shows the duration of the next ISI as a function of the preceding ISI, one can see the presence of bursts in the recordings. By definition a burst is a period of rapid firing, preceded and followed by a longer period of no or low activity. The accumulation of points in the lower left corner indicates that there are numerous periods of rapid firing, which are typical for firing intervals within bursts. The clusters of points in the upper left and lower right corner show that there are also many short ISIs preceded or followed by longer intervals, which indicate the onsets or offsets of bursts. The intervals between bursts display no clear pattern, but the histogram of inter-burst intervals in Figure 2B shows that most intervals are short, and the frequency of longer inter-burst intervals decays. In Figure 2C we plotted the bursts contained in one minute of recordings in the original order in which they appeared. Looking only at the raw data it is not PLoS ONE 2 June 2012 Volume 7 Issue 6 e37354

3 Figure 1. Experimental arrangement for long-term recordings of single cell activity in the tropical rainforest. A) Illustration of the experimental arrangement. The inset shows the morphology of the cell within the prothoracic ganglion after intracellular dye injection (upper part), and a prototype of the portable preparation. B) Illustration of the five stimulus classes played to bushcrickets during experiments. 1) Single pulse of 10 ms; 2) double pulse with 10 ms duration each, separated by an interval of 10 ms; 3) 30 ms pulse; 4) four repetitive pulses, 10 ms each, separated by an interval of 10 ms; 5) 70 ms pulse. C, D) Examples of recordings made at about one hour before sunset (C), and 45 minutes after sunset (D), when the background noise level had increased from 40 db SPL to 65 db SPL. Note that in the low noise situation only a stimulus (arrow) elicited a short burst of spikes, whereas after sunset the neuron fires many bursts also in response to the acoustic background. doi: /journal.pone g001 immediately clear which bursts belong to a common cluster, and although the bursts are from relatively close time points, there is no visible structure of bursts in response to environmental noise. Characteristics of Acoustic Discrimination in Natural Habitats In the recording shown in Figure 1C, each burst of action potential activity before sunset was caused by a stimulus. A detection criterion based on bursts of action potentials or the corresponding increase in spike rate would give hits in term of signal detection [25]. Indeed, in all cases when there was an acoustic signal during the experiment at hrs, there was bursting activity in the nerve cell and there was no, or only single spike spontaneous activity when a signal was absent, therefore there were no misses or false alarms respectively. PLoS ONE 3 June 2012 Volume 7 Issue 6 e37354

4 Figure 2. Analysis of bursts extracted from the spike data. A) The joint interspike-interval plot for a single preparation indicates the presence of bursts by a large cluster of points in the lower left corner, which represents periods of fast firing, and clusters of points in the upper left and lower right corner, which indicate onsets and offset of bursts. B) Histogram of inter-burst intervals (bin size: 0.5 s). C) 13 bursts extracted from 1 minute of the recording. The set of bursts contains 2 responses to stimulus 3 (bursts 10 and 13), 2 responses to stimulus 4 (bursts 4 and 8), and 9 responses to different sources of environmental noise. D) Detection of bursts in spike trains. The 6 spikes in the shaded area constitute a burst, because they are separated by time window of at least 60 ms from the first spike, the interspike-interval is never larger than 30 ms, the burst duration is longer than 8 ms and there are more than 5 spikes. doi: /journal.pone g002 After sunset, however, this ideal situation for signal detection changed due to the strong increase in background noise. The same preparation at exactly the same position in the rainforest now exhibited high action potential activity (Figure 1D), and only an a priori knowledge of the time of signal presentation (arrow) would allow correct detection of the stimulus. Using the same detection criterion as in the situation before sunset would result in many false alarms (i.e. identifying background noise as signals). Figure 3 shows firing and bursting rates in one recording over a longer time period after sunset, and it is obvious that while the recordings are very stable over multiple hours, there are considerable fluctuations on shorter time scales, mainly due to background noise. In this preparation, firing rates vary from 7 Hz to 17 Hz over the time period of 200 minutes of recording, and burst rates vary from about 0.2 to 0.8 Hz. The curves for firing and bursting rates are visibly correlated (correlation coefficient r~0:37). Artificial stimuli are only played every 10 seconds, and preparations typically respond with a single burst to these signals. From the fact that the bursting rate is always greater than 0.1 Hz, one can see that a large majority of the bursts result from background noise (in this recording 80:5% of the bursts are noise bursts). Apparently, analyzing neural signals recorded under natural conditions poses different challenges in comparison to laboratory experiments, but yields different and more realistic results. The background noise mainly constitutes the communication activity of different individuals and species of insects, frogs and vertebrates. In our recordings we made sure to place the preparation at a place in the rainforest where no conspecific males were singing. The majority of noise therefore comes from heterospecifics with no behavioral relevance. A second category of noise may be sound originating from predators, such as bats. Another additional difficulty arises because different noise stimuli arrive at the insect from multiple directions in the azimuth and elevation, due to the complex 3-dimensional structure of the rainforest. Since there is a multitude of simultaneously active senders, the animal receives a variety of sound events, and not all PLoS ONE 4 June 2012 Volume 7 Issue 6 e37354

5 Figure 3. Firing and bursting rates in the natural habitat. Firing (top) and bursting rates (bottom) of the spike activity of the omega-neuron from hrs to 0.40 hrs at night in the natural habitat. The fluctuation in both rates is high, but firing and burst rates are correlated with a correlation coefficient of r~0:37. The mean firing rate over the entire night is 11.5 Hz, and the mean bursting rate is 0.33 Hz. (Bin size: 1 sec for firing rate, 100 sec for bursting rate). doi: /journal.pone g003 of them are coherent in the time and frequency domain. This is very different from typical lab experiments, in which a single stimulus and eventual background noise are broadcast from the same or opposite sides of the animals. Stereotyped Response of the Omega Neuron to Acoustic Stimulation The structure of bursts in response to the artificial stimulus classes from Figure 1B becomes visible if one aligns the bursts to the onset of the stimulus. For a single preparation, Figure 4 shows the bursts that follow these stimuli as spike train plots and PSTHs, respectively. Considering that the environmental noise before and during the presentation of the stimulus is very strong and inhomogeneous, the responses of the omega neuron to the same stimulus are remarkably similar. The same holds for responses of different preparations from different sessions to identical stimuli, which is illustrated in Figure 5. In those recording sessions from November 2003, only stimuli from class 1,2, and 4 were presented. Also here one can observe that the stimulus aligned firing patterns are qualitatively very similar, with slight variations in the latency of the bursts, or the variability of firing. Identification of Burst Patterns with Unsupervised Clustering After extracting all bursts from the recordings, we used a variant of the spike-time metric by Victor and Purpura [15 17] to compute similarities between different bursts (see 6A, B and Materials and Methods). This similarity measure then served as the basis for clustering spike trains into homogeneous groups, and assigning a representative cluster exemplar to every group, using the affinity propagation algorithm [18]. This procedure is completely unsupervised, based only on the similarity of bursts, and not on labels assigned to the bursts. It creates a variable number of groups of bursts, with the goal of maximizing the similarity of bursts (with respect to the spike-time metric) within each group, and minimizing the similarity between different groups. The algorithms are described in detail in the Materials and Methods section. For the first experiment we used recordings in which five artificial stimuli were broadcast to the preparations in their natural habitat after sunset. The five artificial stimuli used for playbacks differed in duration and temporal structure (see Figure 1B). Bursts that occurred at the time of the onset of the artificial stimulus were labeled with the class of the associated stimulus. The labels of the bursts were only used for evaluation purposes, but were not provided to the clustering algorithm. The result of the clustering is illustrated in Figure 6C and D. Here the distance matrix is shown before and after the clustering process (dark indicates large distance). Before the clustering, the distance matrix for 1000 randomly picked bursts is displayed, where the ordering of the burst indices corresponds to the order in which they were recorded. The clustering process rearranged the order of the bursts, by grouping them into homogeneous clusters, which are displayed in the arbitrary order that is produced by the affinity propagation algorithm (see Materials and Methods). One can clearly recognize these groups from the distance matrix, by observing blocks that have low inter-cluster distance, and larger distance to other groups of bursts. Burst Patterns in Response to Artificial Stimuli and Noise Figure 7 shows the groups of bursts resulting from unsupervised clustering, plotting bursts following stimuli in red, and bursts as a result of background noise in black. On the right side we plot a sketch of the stimulus that is assigned as label to this cluster (or N if the cluster consists mostly of noise bursts). This label was determined as the stimulus class with the highest percentage of bursts in this cluster relative to its average frequency of occurrence in the whole dataset (see Materials and Methods). This avoids a bias towards assigning clusters to noise stimuli, which are four times as frequent as bursts in response to the 5 artificial stimuli. The labels of some clusters are very homogeneous, in particular those in the upper part of the dendrogram with clusters of long and relatively unstructured bursts, which are almost exclusively bursts in response to background noise. These bursts are grouped together because they have a similar mean firing rate and a similar number of spikes, although their firing patterns do not exactly match. Two other groups of homogeneous clusters are comprised PLoS ONE 5 June 2012 Volume 7 Issue 6 e37354

6 Probing Real Sensory Worlds of Receivers Figure 4. Stimulus aligned responses for a single preparation. A) Sketch of the five artificial stimuli. B) The structure of the burst spike trains in response to artificial stimuli becomes visible if they are aligned to the stimulus onset. C) Peri-stimulus time histograms (bin size: 2 ms) for the five classes of artificial stimuli. doi: /journal.pone g004 a different preparation and night, and again it can be seen that bursts following one of the longer or more structured artificial signals (classes 4 and 5) fall into more homogeneous clusters than bursts after stimuli with shorter pulses. Bursts following stimulus 1 are again mostly clustered together with noise bursts. A possible explanation for this is that stimuli 1 3 are similar to signals that can be naturally found in the acoustic background noise of the rainforest, e.g. calling songs produced by other bushcrickets, whereas stimuli 4 and 5 are purely artificial and never occur in the background. One can further observe in both plots that there are some very homogeneous clusters of bursts with precisely timed firing patterns in response to unidentified almost exclusively of bursts in response to two artificial stimuli (classes 4 (a four-pulse-stimulus) and 5 (a pulse of 70 ms duration) in Figure 1B); only rarely do we find in the same cluster bursts not elicited by these stimuli. In some cases, however, the unsupervised clustering algorithm produced inhomogeneous clusters, which include both bursts in response to stimuli as well as bursts in response to the background noise. This is true for the two bottom clusters in Figure 7A, which contain most of the bursts in response to the 10 ms pulse (stimulus 1), but also for two clusters with bursts in response to the 30 ms pulse (stimulus 3) and the two-pulse stimulus (stimulus 2), which cluster together with background noise bursts. Figure 7B shows a clustering for another recording with Figure 5. Bursts in response to three artificial stimuli. Recordings are from three different insect preparations (marked by different colors), and bursts are displayed aligned to the stimulus onset. Only classes 1,2, and 4, were played at those recording dates. One can see a clear similarity of the responses, but also different latencies and variabilities of firing. doi: /journal.pone g005 PLoS ONE 6 June 2012 Volume 7 Issue 6 e37354

7 Figure 6. Illustration of modified spike metrics, and spike train distance matrices. A, B) Illustration of the burst-shift operator. In A, the two bursts appear to be very different under the standard spike-time metric from [15], because there is a single (purple) spike before the pattern of two spike triplets, and so all subsequent spike times are shifted. The burst-shift operator in B deletes the first (purple dashed) spike and re-aligns the new first spikes of the two bursts. The distance between the two spike train then results from a relatively cheap series of spike shifts (green dotted lines), plus the cost for deleting the initial spike. C, D) Distance matrices for a single preparation in the natural habitat before (C) and after clustering (D). Light pixels indicate high similarity of bursts, whereas dark pixels show larger distances. The clustering process leads to the clear formation of groups of similar bursts. doi: /journal.pone g006 background noise events. Altogether this illustrates the difficulty of the auditory discrimination problem for the bushcricket under real world conditions. Short and/or unstructured stimuli may lead to false alarms from background noise signals. On the other hand, the discriminability of stimuli can be greatly improved by using more complex temporal structure, such as the patterns in stimulus classes 2 and 4. Our current method finds individual burst patterns that could serve as basic blocks for encoding more complex stimuli, or features of stimuli, in sequences of bursts, which will be an important topic for future research. Occasionally, bursts for the same classes of stimuli are distributed into different clusters (e.g. class 1 in Fig. 7A and B, class 3 in Fig. 7B, or class 5 in Fig. 7B). Since the clustering algorithm does not know about the labels, it does not attempt to avoid this effect, if it can be explained by the model. Therefore it is possible that the same stimulus leads to a single cluster in one experiment, and two or even more in another. The biological interpretation of this effect might be that different variants of the same stimulus can be encoded by different clusters, e.g. due to different background noise during the presentation of the stimulus. However, we did not record the acoustic background during the experiments, since this is technically very difficult to achieve in a real-world environment with complex 3-dimensional structure like the rainforest, and a direct mapping between sound recordings and the acoustic stimulus sensed by the animal is in general not possible. Discriminability of Artificial Stimulus Classes Figure 7 demonstrates that bursts in response to particular classes of artificial stimuli form very homogeneous clusters, e.g. bursts in response to the four pulse pattern (class 4). On the other hand, some stimulus evoked bursts are mostly mixed together with bursts in response to background noise in the habitat, or bursts in response to different stimuli. We evaluated this separability property of stimulus evoked bursts over data from six recordings sessions, of which three used all 5 stimulus classes, and three contained only stimuli of classes 1,2, and 4. As a measure of homogeneity we used the average conditional entropy of class labels for every cluster (see Materials and Methods). The conditional entropy in our case is low if knowing the cluster PLoS ONE 7 June 2012 Volume 7 Issue 6 e37354

8 Probing Real Sensory Worlds of Receivers PLoS ONE 8 June 2012 Volume 7 Issue 6 e37354

9 Figure 7. Clusters of bursts from two recordings in the natural habitat. A) and B) show clusters of bursts obtained from two different preparations on different recording dates. Bursts associated with artificial stimuli are plotted red, bursts associated with noise are plotted in black. On the right is an illustration of the stimulus that is assigned as label to this cluster, or N if the cluster mainly consists of bursts is response to noise signals. The clusters are arranged hierarchically, grouping clusters with similar exemplars together. Longer and more structured bursts form more homogeneous groups than bursts after short pulse signals (e.g. clusters at the bottom of diagram B). Clusters in A) contain between 140 and 501 bursts, and between 145 and 328 bursts in B). doi: /journal.pone g007 label reduces the uncertainty about the classes of bursts that are found within the cluster. So in the ideal case there should be only one class of bursts in every cluster (which means zero entropy). Figure 8A shows the average conditional entropy individually for every class, where the average is over all recordings sessions in which those stimuli were used (three sessions for all 5 stimuli, three sessions only for stimulus classes 1,2, and 4). Although these statistics are based on only six recording sessions, and the standard deviations are large, one can observe the same trend that was qualitatively visible from Figure 7. The average conditional entropy is low for classes of bursts in response to long and/or temporally structured stimuli (classes 2,4, and 5), and higher for the single short pulses of classes 1 and 3. This indicates that classes 2,4, and 5 can be better discriminated from other artificial or background signals than the single pulse stimuli. Due to the limited amount of available data, these results are statistically not significant, and more measurements would be required. In Figure 8B we show the confusion matrix that results from assigning class labels to each cluster (see Materials and Methods). One can see that the major source of errors are noise bursts being assigned to clusters that represent artificial stimulus classes. Classes 2 and 3 are also sometimes clustered together, whereas classes 4 and 5 are mostly found in homogeneous clusters. Similarity of Burst Patterns in Different Preparations The two clustering results in Figures 7A and 7B indicate that similar clusters of bursts can be found in both recordings, even though the recordings stem from different preparations and different recording sessions. This is even more remarkable if one considers that the background noise in the natural habitat is far from constant over the recording periods, since different sound sources may be present and located at different positions in comparison to other recording sessions in different nights, and even years. To further analyze this similarity of neural responses, we searched for burst clusters from one recording session that have corresponding clusters of bursts in different recording sessions with similar firing patterns. Starting from the previously computed clusterings of bursts from single recording sessions, the spike-time metric described in Methods was used to calculate all distances between the cluster exemplars from different sessions. For every cluster in one recording the cluster with the closest matching exemplar in the other recording was then selected. Since we have observed that firing patterns are characteristic for the acoustic stimulus that they encode, a high similarity of two clusters in different sessions would suggest that the contained bursts are responses to the same or a similar type of sound source. For comparison, and for understanding whether the matching is based on the firing pattern or purely on statistical properties like average firing rate and duration, we also computed the average spike-time metric between bursts from the two clusters if every spike train was replaced by an inhomogeneous Poisson spike train, whose time-dependent firing rate profile was given by the average population rate of all bursts in the cluster. In all experiments we observed that the distances Figure 8. Analysis of separability of bursts in response to artificial stimuli from other stimulus classes. A) Conditional entropy of class labels given the cluster indices, averaged over six recording sessions, three in which all 5 stimulus classes were used, and three in which only classes 1,2, and 4 were used (errorbars denote standard deviations). Classes of bursts with lower conditional entropy form more homogeneous clusters. Artificial stimuli that consist of temporally more structured and/or longer stimuli (classes 2,4, and 5) are better separable from noise or other stimuli than single pulse stimuli (classes 1 and 3). B) Confusion matrix for assigned cluster labels vs. actual labels of bursts. In every row we plot the average relative frequencies of burst labels occurrences in clusters that were assigned to one of the classes N ( = Noise) or 1 5. One can see that most mistakes are due to noise bursts assigned to one of the artificial stimulus classes. Also bursts in response to classes 2 and 3 are sometimes clustered together. doi: /journal.pone g008 PLoS ONE 9 June 2012 Volume 7 Issue 6 e37354

10 between the Poisson spike trains were always substantially higher than the matching distances of the cluster exemplars. This indicates that the firing patterns of bursts inside a cluster are much more precise than the Poisson spike trains. We first analyze the similarity of clusters in response to the same stimuli under two different acoustic conditions. For several preparations we recorded the response to artificial stimuli in the laboratory, and for others outdoors. Obviously these recording conditions are very different, because the majority of bursts (around 80%) in outdoor recordings stem from background noise, while in the laboratory bursts occur almost exclusively in response to artificial stimuli (only 2:7% of all bursts result from spontaneous activity). In Figure 9 the clusters of bursts found in a laboratory experiment, in which only artificial stimuli of classes 1,3, and 4 (see Figure 1B) were broadcast, were matched to clusters from outdoor recordings (the burst labels are not used for the matching). As can be seen from the comparison of clusters, there are very close matches of laboratory-burst clusters to clusters from outdoor recordings. On the other hand, the responses to the four-pulsed stimulus in the laboratory condition reveal a more precise timing of spikes within the bursts compared to the responses recorded outdoors. This indicates that the specific acoustic conditions of the noisy nocturnal rainforest caused some changes in this timing within bursts. Still, even in the presence of this strong distracting noise, the omega neuron responds with a very similar pattern, that significantly simplifies the decoding tasks for higher processing areas. In a similar way we matched clusters of bursts from different outdoor recording sessions, in which the activity of omega neuron from different animals was recorded at different nights (sometimes in different years). The examples of cluster matching results in Figure 10 show that also under these conditions it is possible to find close matches for some clusters of bursts. Comparing the similarity indices D in Figures 9 and 10 would indicate that some of the cluster matches between different animals in different outdoor recording conditions are closer than the cluster matches between outdoor and laboratory recording conditions. The reason for this might be the lower number of spikes under lab conditions, due to the complete absence of noise. The matching distance is substantially lower than the distance of Poisson spike trains with identical statistics, which indicates that the precision of firing in both preparations is higher than can be explained by a stochastic, purely firing-rate based model. On the other hand, the cluster matching procedure also revealed several clusters of bursts for which no good match in the other recording session was found. This holds in particular for clusters of long bursts without clear temporal structure, which typically result from background noise. Such clusters have a substantially higher distance D to the best matching cluster, which is due to the fact that these bursts include more spikes, and so more shifts or insertions may have to be made in order to transform one spike train into another. Bursts within these clusters do not show the precise temporal signature that could be observed in the previous analysis, and could arise e.g. in response to senders in the background with temporally extended calls and low amplitude modulation. Such long stimuli also have a higher probability of being interrupted by another stimulus of higher behavioral relevance. It is therefore not unexpected to find higher variability in these burst patterns, both within the same preparation and between different preparations. Similarity of Burst Patterns in Simultaneous Recordings of Homologous Cells In the previous section we compared the similarity of burst activity in the omega-neuron between lab and outdoor recordings, or between different cells in different nights. The biological microphone approach offers in addition one unique opportunity Figure 9. Matching of laboratory burst clusters to outdoor recordings. Clusters of bursts from laboratory recordings (black), matched to clusters of bursts from outdoor recordings (red). Four examples of matched clusters (in response to stimulus classes 4 (top left), 3 (top right), and 1 (bottom left and right) are shown. For these laboratory clusters, closely matching clusters are found in the outdoor recordings. D defines the distance between the exemplars of the two matched clusters under the spike-time metric. As a comparison, the numbers in parentheses give the average distances between Poisson spike trains with identical time-varying firing rate profiles. doi: /journal.pone g009 PLoS ONE 10 June 2012 Volume 7 Issue 6 e37354

11 Figure 10. Clusters of bursts from two different outdoor recordings and their best matching cluster. The examples show matches for burst clusters in response to natural background noise. D defines the distance between the exemplars of the two matched clusters under the spiketime metric. The numbers in parentheses give the average distances between Poisson spike trains with identical time-varying firing rate profiles. doi: /journal.pone g010 to test the power of our method, by comparing the burst responses of omega cells from two different preparations recorded simultaneously, and placed next to each other, so that they experience the same acoustic events. For the experiment presented in Figures 11 and 12 the two preparations were placed in the nocturnal rainforest, at a distance of about 10 cm from each other, so that they were exposed to the same acoustic environment. Prior to these recordings, the threshold of each omega-cell in response to a pure tone, 20 khz stimulus was determined in the laboratory, and one preparation was 5 db less sensitive compared to the other preparation. In this experiment, no artificial sound stimuli were broadcast to the preparations, so all bursts had been elicited as a result of background noise alone. Figure 11A shows a short sequence of the original spike recording of both cells, and in Figure 11B the firing and burst rates of both cells are illustrated for a sequence of continuous 20 minutes of recording. The gross firing and bursting pattern of both cells is rather similar (Figure 11A), although the less sensitive cell exhibits a reduced firing rate (Figure 11B). The firing rates are actually correlated with a correlation coefficient of r~0:34, the bursting rates are correlated with r~0:27. Even though the firing behavior of the two omega-cells is slightly different due to the threshold difference of 5 db, one should find similar spiking patterns in the bursts, as the two preparations had been exposed to the same background noise. For both preparations the bursts were extracted, which results in 936 bursts for preparation 1 and 726 for preparation 2. We used the affinity propagation algorithm to find clusters in the aggregated set of bursts from both preparations. Figure 12A shows the resulting cluster dendrogram, where bursts from the first preparation are drawn in red, and bursts from the second preparation in black. Every cluster contains bursts from both preparations, and the relative frequencies of bursts originating from either preparation are balanced. On average, 55:8% of the bursts in every cluster are from preparation 1, which is a result of the higher number of bursts extracted from preparation 1. The minimum percentage of bursts from preparation 1 in any cluster is 28:5%, and the maximum percentage is 73:3%. We also show in Figure 12B for every burst in the two preparations the spiketime distance of the closest matching burst in the same, and in the other preparation. One can see that those distances scatter around the diagonal, which indicates that for every firing pattern in one preparation we can find a similar one in the other, which is not a worse match than other bursts in the same preparation. If one looks at the exact spike times of the two preparations, one can see in Figure 12C the relative frequency that within a time window of Dt before and after a spike in a burst in one preparation there is a spike in a burst in the other preparation. In more than 20% of the cases there is a spike within 1 ms, and in 80% of all cases there is a spike within 10 ms. This shows that even though the two preparations do not fire at exactly the same time, they will frequently fire within a short time window after each other. All these results suggest that there are no firing patterns that are uniquely found only in one preparation, but not in the other. Even though individual burst responses of the two preparations at any time may show stronger variations, the global bursting patterns in response to the same acoustic background are very similar for different preparations. This result provides further evidence for the remarkably well preserved burst-coding mechanism in response to complex real-world auditory stimuli that is shared by individuals of this species. In the following Discussion we will analyze the significance of these results for communication under real-world conditions. PLoS ONE 11 June 2012 Volume 7 Issue 6 e37354

12 Probing Real Sensory Worlds of Receivers Figure 11. Firing and bursting during simultaneous recordings. A) Short sequence of the original spike recording of both cells recorded simultaneously. B): Firing and burst rates of both cells for a duration of 20 minutes. The firing rates of preparation 1 and 2 are correlated with a correlation coefficient of r~0:34. The burst rates are correlated with r~0:27. Mean firing rates over the entire 20 minute recordings are 9.98 Hz (preparation 1) and Hz (preparation 2), and mean bursting rates are 0.53 Hz and 0.41 Hz respectively. (Bin size: 1 sec for firing rate, 100 sec for bursting rate). doi: /journal.pone g011 context of the efficiency of synaptic transmission and thereby synaptic plasticity [7], in the selective distribution of information to different target neurons [26], or the dynamics of encoding behaviorally relevant stimulus features [4,27 30]. Bursts can be viewed as robust symbols for the neural coding alphabet; they can carry information in their duration, the number of spikes, or the exact timing of the firing pattern, and specifically tuned synapses may read out such a code easily [31]. However, classification and discrimination are severely impaired by variation in afferent spike trains, either as a result of Discussion Coding Problems for Stimuli in the Natural Environment For the two major tasks of sensory systems of object classification and discrimination the central nervous system needs to interpret the ongoing afferent spike activity. Consistent with a number of previous studies on sensory coding in different modalities we view short bursts of action potentials as the basic units for the representation of information. The importance of bursts, in contrast to single spikes, has been discussed in the PLoS ONE 12 June 2012 Volume 7 Issue 6 e37354

13 Figure 12. Similarity of burst patterns in two simultaneously recorded preparations. A) Clusters for the aggregated bursts of two omegacell preparations recorded simultaneously in their natural habitat; no broadcast of artificial stimuli. Bursts from preparation 1 are drawn in red, and bursts from preparation 2 in black. Every cluster contains about half of its bursts from one preparation. B) Spike-time distance of the most similar burst in the same preparation in comparison to the most similar burst in the other preparation. The clustering of points around the diagonal shows that for every burst in one preparation the distance of the closest match in the same and in the other preparation are almost identical. C) Similarity of spiking times within bursts. For every spike at time t in a burst in one preparation we compute for different time windows Dt how often there is a spike within a burst in the other preparation in the time window ½t{Dt,tzDtŠ. We find that 65% of all spikes have a corresponding spike in the other preparation within a 5 ms time window, and 80% within a 10 ms time window. doi: /journal.pone g012 intrinsic noise in nervous systems, or external noise resulting from interactions of the stimulus with the transmission channel. A further source of variability of high relevance for a receiver is introduced as a result of small differences in the features of signals from different sources, such as the signals of mates. Ronacher et al. [9] reviewed the sources of spike train variability and the associated problems and constraints for producing adaptive behavior in grasshoppers. In the case of the auditory system, a further problem results from the background noise of many natural environments, so that relevant stimuli (and stimulus variants) have to be discriminated from irrelevant background noise. Research in the past decade demonstrated that the auditory system of many animals evolved mechanisms to cope with such noise [32]. In all comparable studies of neural coding with bursts in the past, preparations were studied under controlled conditions in the sound proof lab. The studies by [33] in grasshoppers and [34,35] in songbirds, for example, played back previously recorded songs of conspecifics, and investigated how the individual songs can be discriminated from the neural response of auditory receptor cells [33] or cortical neurons [34,35]. Another common approach to study neural coding is to use time-varying (often random) artificial stimuli, and measure how accurately the whole stimulus, or certain features of the stimulus, can be reconstructed from the neural response in-vitro or in-vivo [27 30,36,37]. The main advantage of these methods is that the experimenter has full control over the stimuli (e.g. to modify their duration or amplitude), and eliminates distractor signals. On the other hand, the complete absence of environmental noise creates an artificial scenario for the receiver, which may hide the influence of potentially important selective attention mechanisms [38,39]. In contrast to these previous studies we investigate here the most realistic possible scenario for sensory coding, using stimuli and neurophysiological recordings directly in the natural habitat of the organism. We did not attempt to correlate the bursts in the omega neuron with simultaneously recorded sound stimuli, since it is PLoS ONE 13 June 2012 Volume 7 Issue 6 e37354