Abstract musical timbre and physical modeling

Transcription

1 Abstract musical timbre and physical modeling Giovanni De Poli and Davide Rocchesso June 21, 2002 Abstract As a result of the progress in information technologies, algorithms for sound generation and transformation are now ubiquitous in multimedia systems, even though their performance and quality is rarely satisfactory. For the specific needs of music production and multimedia art, sound models are needed which are versatile, responsive to user s expectations, and having high audio quality. Moreover, for human-machine interaction model flexibility is a major issue. We will review some of the most important computational models that are being used in musical sound production, and we will see that models based on the physics of actual or virtual objects can meet most of the requirements, thus allowing the user to rely on high-level descriptions of the sounding entities. 1 Introduction In our everyday experience, musical sounds are increasingly listened to by means of loudspeakers. On the one hand, it is desirable to achieve a faithful reproduction of the sound of acoustic instruments in highquality auditoria. On the other hand, the possibilities offered by digital technologies should be exploited to approach sound-related phenomena in a creative way. Both of these needs call for mathematical and computational models of sound generation and processing. The timbre produced by acoustic musical instruments is caused by the physical vibration of a certain resonating structure. This vibration can be described by signals that correspond to the time-evolution of the acoustic pressure associated to it. The fact that the sound can be characterized by a set of signals suggests quite naturally that some computing equipment could be successfully employed for generating timbres, for either the imitation of acoustic instruments or the creation of new sounds with novel timbral properties. The focus of this article is on computational models of timbre generation, especially on those models that are directly based on physical descriptions of generation process. The general framework of sound modeling is explained in sections 2 and 3. We will divide modeling paradigms into signal models and physics-based models. They will be illustrated in Section 5 and 6. Several techniques are presented for modeling sound sources and general, linear and nonlinear acoustic systems. 2 Computational models as abstract timbre models In order to generate, manipulate, and think about timbre, it is useful to organize our intuitive sound abstractions into sound objects, in the same way as abstract categories are needed for defining visual objects. The first extensive investigation and systematization of sound objects from a perceptual viewpoint was done by Pierre Shaeffer in the fifties [1]. Nowadays, a common terminology is available for describing sound objects both from a phenomenological or a referential viewpoint, and for describing collections of such objects (i.e. soundscapes) [2], [3], [4]. For effective generation and manipulation of the timbre it is necessary to define models for sound synthesis, processing, and composition. Identifying models, either visual or acoustic, is equivalent to making 1

2 high-level constructive interpretations, built up from the zero level (i.e. pixels or sound samples). It is important for the model to be associated with a semantic interpretation, in such a way that an intuitive action on model parameters becomes possible. A sound generation model is implemented by means of sound synthesis and processing techniques. A wide variety of sound synthesis algorithms is currently available either commercially or in the literature. Each one of them exhibits some peculiar characteristics that could make it preferable to others, depending on goals and needs. Technological progress has made enormous steps forward in the past few years as far as the computational power that can be made available at low cost is concerned. At the same time, sound synthesis methods have become more and more computationally efficient and the user interface has become friendlier and friendlier. As a consequence, musicians can nowadays access a wide collection of synthesis techniques (all available at low cost in their full functionality), and concentrate on their timbral properties. Each sound synthesis algorithm can be thought of as a computational model for the sound itself. Though this observation may seem quite obvious, its meaning for sound synthesis is not so straightforward. As a matter of fact, modeling sounds is much more than just generating them, as a computational model can be used for representing and generating a whole class of sounds, depending on the choice of control parameters. The idea of associating a class of sounds to a digital sound model is in complete accordance with the way we tend to classify natural musical instruments according to their sound generation mechanism. For example, strings and woodwinds are normally seen as timbral classes of acoustic instruments characterized by their sound generation mechanism. It should be clear that the degree of compactness of a class of sounds is determined, on one hand, by the sensitivity of the digital model to parameter variations and, on the other hand, the amount of control that is necessary to obtain a certain desired sound. As an extreme example we may think of a situation in which a musician is required to generate sounds sample by sample, while the task of the computing equipment is just that of playing the samples. In this case the control signal is represented by the sound itself, therefore the class of sounds that can be produced is unlimited but the instrument is impossible for a musician to control and play. An opposite extremal situation is that in which the synthesis technique is actually the model of an acoustic musical instrument. In this case the class of sounds that can be produced is much more limited (it is characteristic of the mechanism that is being modeled by the algorithm), but the degree of difficulty involved in generating the control parameters is quite modest, as it corresponds to physical parameters that have an intuitive counterpart in the experience of the musician. An interesting conclusion that could be already drawn in the light of what we stated above is that the generality of the class of sounds associated to a sound synthesis algorithm is somehow in contrast with the playability of the algorithm itself. One should remember that the playability is of crucial importance for the success of a specific sound synthesis algorithm as, in order for a sound synthesis algorithm to be suitable for musical purposes, the musician needs an intuitive and easy access to its control parameters during both the sound design process and the performance. Such requirements often represents the reason why a certain synthesis technique is preferred to others. From a mathematical viewpoint, the musical use of sound models opens some interesting issues: description of a class of models that are suitable for the representation of musically-relevant acoustic phenomena; description of efficient and versatile algorithms that realize the models; mapping between meaningful acoustic and musical parameters and numerical parameters of the models; analysis of sound signals that produces estimates of model parameters and control signals; approximation and simplification of the models based on the perceptual relevance of their features; generalization of computational structures and models in order to enhance versatility. 3 Sound Modeling In the music sound domain, we define generative models as those models which give computational form to abstract objects, thus representing a sound generation mechanism. 2

3 Generative models can represent the dynamics of real or virtual generating objects (physics-based models), or they can represent the physical quantities as they arrive to human senses (signal models) [5] (see figure??). In our terminology, signal models are models of signals as they are emitted from loudspeakers or arrive to the ears. The connection with human perception is better understood when considering the evaluation criteria of the generative models. The evaluation of a signal model should be done according to certain perceptual cues. On the contrary, physics-based models are better evaluated according to the physical behaviors involved in the sound production process. In classic sound synthesis, signal models dominated the scene, due to the availability of very efficient and widely applicable algorithms (e.g. frequency modulation). Moreover, signal models allow to design sounds as objects per se without having to rely on actual pieces of material which act as a sound source. However, many people are becoming convinced of the fact that physics-based models are closer to the users/designers needs of interacting with sound objects. The semantic power of these models seems to make them preferable for this purpose. The computational complexity of physically-based algorithms is becoming affordable with nowadays technology, even for real-time applications. We keep in mind that the advantage we gain in model expressivity comes to the expense of the flexibility of several general-purpose signal models. For this reason, signal models keep being the model of choice in many applications, especially for music composition. In the perspective of a multisensorial unification under common models, physics-based models offer an evident advantage over signal models. In fact, the mechanisms of perception for sight and hearing are very different, and a unification at this level looks difficult. Even though analogies based on perception are possible, an authentic sensorial coherence seems to be ensured only by physics-based models. The interaction among various perceptions can be an essential feature if we want to maximize the amount of information conveyed to the spectator/actor. The unification of visual and aural cues is more properly done at the level of abstractions, where the cultural and experience aspects become fundamental. Thus, building models closer to the abstract object, as it is conceived by the designer, is a fundamental step in the direction of this unification. 4 Classic Signal Models Here we will briefly overview the most important signal models for musical sounds. A more extensive presentation can be found in several tutorial articles and books on sound synthesis techniques [6], [7], [8], [9], [10], [11]. Instead, section 5 will cover the most relevant paradigms in physically-based sound modeling. 4.1 Spectral models Since the human ear acts as a particular spectrum analyser, a first class of synthesis models aims at modeling and generating sound spectra. The Short Time Fourier Transform and other time-frequency representations provide powerful sound analysis tools for computing the time-varying spectrum of a given sound Sinusoidal model When we analyze a pitched sound, we find that its spectral energy is mainly concentrated at a few discrete (slowly time-varying) frequencies f i. These frequency lines correspond to different sinusoidal components called partials. If the sound is almost periodic, the frequencies of partials are approximately multiple of the fundamental frequency f 0, ie. f i (t) i f 0 (t). The amplitude a i of each partial is not constant and its time-variation is critical for timbre characterization. If there is a good degree of correlation among the frequency and amplitude variations of different partials, these are perceived as fused to give a unique sound with its timbre identity. 3

4 The sinusoidal model assumes that the sound can be modeled as a sum of sinusoidal oscillators whose amplitude a i and frequency f i are slowly time-varying s s (t) = i a i (t) cos[φ i (t)], (1) or, digitally, φ i (t) = t 0 2πf i (τ)dτ + φ i (0), (2) s s (n) = i a i (n) cos[φ i (n)], (3) φ i (n) = 2π f i (n)t s + φ i (n 1), (4) where T s is the sampling period. Equations (1) and (2) are a generalization of the Fourier theorem, that states that a periodic sound of frequency f 0 can be decomposed as a sum of harmonically related sinusoids s s (t) = i a i cos(2πif 0 t + φ i ). This model is also capable of reproducing aperiodic and inharmonic sounds, as long as their spectral energy is concentrated near discrete frequencies (spectral lines). In computer music this model is called additive synthesis and is widely used in music composition. Notice that the idea behind this method is not new. As a matter of fact, additive synthesis has been used for centuries in some traditional instruments such as organs. Organ pipes, in fact, produce relatively simple sounds that, combined together, contribute to the richer spectrum of some registers. Particularly rich registers are created by using many pipes of different pitch at the same time. Moreover this method, developed for simulating natural sounds, has become the metaphorical foundation of a compositional methodology based on the expansion of the time scale and the reinterpretation of the spectrum in harmonic structures Random noise models The spread part of the spectrum is perceived as random noise. The basic noise generation algorithm is the congruential method s n = [as n (n 1) + b] mod M. (5) With a suitable choice of the coefficients a and b it produces pseudorandom sequences with flat spectral density magnitude (white noise). Different spectral shapes ca be obtained using white noise as input to a filter Filters Some sources can be modeled as an exciter, characterized by a spectrally rich signal, and a resonator, described by a linear system, connected in a feed-forward relationship. An example is the voice, where the periodic pulses or random fluctuations produced by the vocal folds are filtered by the vocal tract, that shapes the spectral envelope. The vowel quality and the voice color greatly depends on the resonance regions of the filter, called formants. If the system is linear and time-invariant, it can be described by the filter H(z) = B(z)/A(z) that can be computed by a difference equation s f (n) = i b i u(n i) k a k s f (n k). (6) 4

5 where a k e b i are the filter coefficients and u(n) e s f (n) are input and output signals. The model is also represented by the convolution of the source u(n) with the impulse response of the filter s f (n) = (u h)(n) = k h(n k)u(k). (7) Digital signal processing theory gives us the tools to design the filter structure and to estimate the filter coefficients in order to obtain a desired frequency response. This model combines the spectral fine structure (spectral lines, broadband or narrowband noise, etc.) of the input with the spectral envelope shaping properties of the filter: S f (f) = U(f) H(f). Therefore, it is possible to control and modify separately the pitch from the formant structure of a speech sound. In computer music this model is called subtractive synthesis. If the filter is static, the temporal features of the input signal are maintained. If, conversely, the filter coefficients are varied, the frequency response changes. As a consequence, the output will be a combination of temporal variations of the input and of the filter (cross-synthesis). If we make some simplifying hypothesis about the input, it is possible to estimate both the parameters of the source and the filter of a given sound. The most common procedure is linear predictive coding (LPC) which assumes that the source is either a periodic impulse train or white noise, and that the filter is all pole (i.e., no zeros) [12]. LPC is widely used for speech synthesis and modification. A special case is when the filter features a long delay as in s f (n) = βu(n) αs f (n N p ). (8) This is a comb type filter featuring frequency resonances multiple of a fundamental f p = F s /N p, where F s = 1/T s is the sampling rate. If initial values are set for the whole delay line, for example random values, all the frequency components that do not coincide with resonance frequencies are progressively filtered out until a harmonic sound is left. If there is attenuation (α < 1) the sound will have a decreasing envelope. Substituting α and/or β with filters, the sound decay time will depend on frequency. For example if α is smaller at higher frequencies, the upper harmonics will decay faster than the lower ones. We can thus obtain simple sound simulations of the plucked strings [13], [14], where the delay line serves to establish oscillations. This method is suitable to model sounds produced by a brief excitation of a resonator, where the latter establishes the periodicity, and the interaction between exciter and resonator can be assumed to be feedforward. This method is called long-term prediction or Karplus-Strong synthesis. More general musical oscillators will be discussed in sect Time domain models When the sound characteristics are rapidly varying, as during attacks or non stationary sounds, spectral models tend to presents artifacts, due to low time-frequency resolution or to the increase of the amount of data used in the representation. To overcome these difficulties, time domain models were proposed. A first class, called sampling or wavetable, stores the waveforms of musical sounds or sound fragments in a database. During synthesis, a waveform is selected and reproduced with simple modifications, such as looping of the periodic part, or sample interpolation for pitch shifting. The same idea is used for simple oscillators, that repeats a waveform stored in a table (table-lookup oscillator) Granular models More creative is the granular synthesis model. The basic idea is that a sound can be considered as a sequence, possibly with overlaps, of elementary and short acoustic elements called grains. Additive synthesis starts from the idea of dividing the sound in the frequency domain into a number of simpler elements (sinusoidal). Granular synthesis, instead, starts from the idea of dividing the sound in the time domain into a 5

6 sequence of short elements called grains. The parameters of this technique are the waveform of the grain g k ( ), its temporal location l k and amplitude a k s g (n) = k a k g k (n l k ). (9) A complex and dynamic acoustic event can be constructed starting from a large quantity of grains. The features of the grains and their temporal locations determine the sound timbre. We can see it as being similar to cinema, where a rapid sequence of static images gives the impression of objects in movement. The initial idea of granular synthesis dates back to Gabor [15], while in music it arises from early experiences of tape electronic music. The choice of parameters can be via various criteria, at the base of which, for each one, there is an interpretation model of the sound. In general, granular synthesis is not a single synthesis model but a way of realizing many different models using waveforms that are locally defined. The choice of the interpretation model implies operational processes that may affect the sonic material in various ways. The most important and classic type of granular synthesis (asynchronous granular synthesis) distributes grains irregularly on the time-frequency plane in form of clouds [16]. The grain waveform is g k (i) = w d (i) cos(2πf k T s i), (10) where w d (i) is a window of length d samples, that controls the time span and the spectral bandwidth around f k. For example, randomly scattered grains within a mask, which delimits a particular frequency/amplitude/time region, result in a sound cloud or musical texture that varies over time. The density of the grains within the mask can be controlled. As a result, articulated sounds cane be modeled and, wherever there is no interest in controlling the microstructure exactly, problems involving the detailed control of the temporal characteristics of the grains can be avoided. Another peculiarity of granular synthesis is that it eases the design of sound events as parts of a larger temporal architecture. For composers, this means a unification of compositional metaphors on different scales and, as a consequence, the control over a time continuum ranging from the milliseconds to the tens of seconds. There are psychoacoustic effects that can be easily experimented by using this algorithm, for example crumbling effects and waveform fusions, which have the corresponding counterpart in the effects of separation and fusion of tones. 4.3 Hybrid models Different models can be combined in order to have a more flexible and effective sound generation. One approach is Spectral Modeling Synthesis (SMS) [17] that considers sounds as composed by a sinusoidal part s s (t) (see Eq. 1), corresponding to the main system modes of vibration, and a residual r(t), modeled as the convolution of white noise with a time-varying frequency shaping filter (see Eq. 7) s sr (t) = s s (t) + r(t). (11) The residual comprises the energy produced in the excitation mechanism which is not transformed into stationary vibrations, plus any other energy contribution that is not sinusoidal in nature. By using the short time Fourier transform and a peak detection algorithm, it is possible to separate the two parts at the analysis stage, and to estimate the time varying parameters of these models. The main advantage of this model is that it is quite robust to sound transformations that are musically relevant, such as time stretching, pitch shifting, and spectral morphing. In the SMS model, transients and rapid signal variations are not well represented. Verma et al. [18] proposed an extension of SMS that includes a third component due to transients. Their method is called Sinusoids+Transients+Noise (S+T+N) and is expressed by s ST N (t) = s s (t) + s g (t) + r(t), (12) 6

7 where s g (t) is a granular term representing the signal transients. This term is automatically extracted from the SMS residual using the Discrete Cosine Transform, followed by a second SMS analysis in the frequency domain. 4.4 Abstract models: frequency modulation Another class of sound synthesis algorithms is neither derived from physical mechanisms of sound production, nor from any sound analysis techniques. These are algorithms derived from the mathematical properties of a formula. The most important of these algorithms is the so called synthesis by Frequency Modulation (FM) [19]. The technique works as an instantaneous modulation of the phase or frequency of a sinusoidal carrier according to the behavior of another signal (modulator), which is usually sinusoidal. The basic scheme can be expressed as follows: s(t) = sin[2πf c t + I sin(2πf m t)] = J k (I) sin[2π(f c + kf m )t] (13) k= where J k (I) is the Bessel function of order k. The resulting spectrum presents lines at frequencies f c ± kf m. The ratio f c /f m determines the spectral content of sounds, and is directly linked to some important features, like the absence of even components, or the inharmonicity. The parameter I (modulation Index) controls the spectral bandwidth around f c, and is usually associated with a time curve (the so called envelope), in such a way that time evolution of the spectrum is similar to that of traditional instruments. For instance, a high value of the modulation index determines a wide frequency bandwidth, as it is found during the attack of typical instrumental sounds. On the other hand, the gradual decrease of the modulation index determines a natural shrinking of the frequency bandwidth during the decay phase. From the basic scheme, other variants can be derived, such as parallel modulators and feedback modulation. So far, however, no general algorithm has been found for deriving the parameters of an FM model from the analysis of a given sound, and no intuitive interpretation can be given to the parameter choice, as this synthesis technique does not evoke any previous musical experience of the performer. The main qualities of FM, i.e. great timbre dynamics with just a few parameters and a low computational cost, are progressively losing importance within modern digital systems. Other synthesis techniques, though more expensive, can be controlled in a more natural and intuitive fashion. The FM synthesis, however, still preserves the attractiveness of its own peculiar timbre space and, although it is not particularly suitable for the simulation of natural sounds, it offers a wide range of original synthetic sounds that are of considerable interest for computer musicians. 5 Physics-based Models In the family of physics-based models we put all the algorithms generating sounds as a side effect of a more general process of simulation of a physical phenomenon. Physics-based models can be classified according to the way of representing, simulating and discretizing the physical reality. Hence, we can talk about cellular, finite-difference, and waveguide models, thus intending that these categories are not disjoint but, in some cases, they represent different viewpoints on the same computational mechanism. Moreover, physics-based models have not necessarily to be based on the physics of the real world, but they can, more generally, gain inspiration from it; in this case we will talk about pseudo-physical models. In this chapter, the approach to physically-based synthesis is carried on with particular reference to real-time applications, therefore the time complexity of algorithms plays a key role. We can summarize the general objective of the presentation saying that we want to obtain models for large families of sounding objects, and these models have to provide a satisfactory representation of the acoustic behavior with the minimum computational effort. 7

8 5.1 Functional Blocks In real objects we can often outline functionally distinct parts, and express the overall behavior of the system as the interaction of these parts. Outlining functional blocks helps the task of modeling, because for each block a different representation strategy can be chosen. In addition, the range of parameters can be better specified in isolated blocks, and the gain in semantic clearness is evident. Our analysis stems from musical instruments, and this is justified by the fact that the same generative mechanisms can be found in many other physical objects. In fact, we find it difficult to think about a physical process producing sound and having no analogy in some musical instrument. For instance, friction can be found in bowed string instruments, striking in percussion instruments, air turbulences in jet-driven instruments, etc.. Generally speaking, we can think of musical instruments as a specialization of natural dynamics for artistic purposes. Musical instruments are important for the whole area of sonification in multimedia environments because they constitute a testbed where the various simulation techniques can easily show their merits and pitfalls. The first level of conceptual decomposition that we can devise for musical instruments is represented by the interaction scheme of figure 1, where two functional blocks are outlined: a resonator and an exciter. The resonator sustains and controls the oscillation, and is related with sound attributes like pitch and spectral envelope. The exciter is the place where energy is injected into the instrument, and it strongly affects the attack transient of sound, which is fundamental for timbre identification. The interaction of exciter and resonator is the main source of richness and variety of nuances that can be obtained from a musical instruments. When translating the conceptual decomposition into a model, two dynamic systems are found [20]: the excitation block, which is strongly non-linear, and the resonator, supposed to be linear to a great extent. The player controls the performance by means of inputs to the two blocks. The interaction can be feedforward, when the exciter doesn t receive any information from the resonator, or feedback, when the two blocks exert a mutual information exchange. In this conceptual scheme, the radiating element (bell, resonating body, etc.) is implicitly enclosed within the resonator. In a clarinet, for instance, we have a feedback structure where the reed is the exciter and the bore with its bell acts as a resonator. The player exert exciting actions such as controlling the mouth pressure and the embouchure, as well as modulating actions such as changing the bore effective length by opening and closing the holes. In a plucked string instrument, such as a guitar, the excitation is provided by plucking the string, the resonator is given by the strings and the body, and modulating actions take the form of fingering. The interaction is only weakly feedback, so that a feedforward scheme can be adopted as a good approximation: the excitation imposes the initial conditions and the resonator is then left free to vibrate. Exciting Actions EXCITER Non-Linear Dynamic System RESONATOR Linear Dynamic System Out Modulating Actions Figure 1: Exciter-Resonator Interaction Scheme In practical physical modeling the block decomposition can be extended to finer levels of detail, as both the exciter and the resonator can be further decomposed into simpler functional components, e.g. the holes and the bell of a clarinet as a refinement of the resonator. At each stage of model decomposition, we are faced with the choice of expanding the blocks further (white-box modeling), or just considering the input-output behavior of the basic components (black-box modeling). In particular, it is very tempting to model just the input-output behavior of linear blocks, because in this case the problem reduces to filter design. However, 8

9 such an approach provides structures whose parameters are difficult to interpret and, therefore, to control. In any case, when the decomposition of an instrument into blocks corresponds to a similar decomposition in digital structures, a premium in efficiency and versatility is likely to be obtained. In fact, we can focus on functionally distinct parts and try to obtain the best results from each before coupling them together [21]. In digital implementations, in between the two blocks exciter and resonator, a third block is often found. This is an interaction block and it can convert the variables used in the exciter to the variables used in the resonator, or avoid possible anomalies introduced by the discretization process. The idea is to have a sort of adaptor for connecting different blocks in a modular way. This adaptor might also serve to compensate the simplifications introduced by the modeling process. To this end, a residual signal might be introduced in this block in order to improve the sound realism. The limits of a detailed physical simulation are also found when we try to model the behavior of a complex linear vibrating structure, such as a soundboard; in such cases it can be useful to record its impulse response and include it in the excitation signal as it is provided to a feedforward interaction scheme. Such a method is called commuted synthesis, since it makes use of commutativity of linear, time-invariant blocks [22], [23]. It is interesting to notice that the integration of sampled noises or impulse responses into physical models is analogous to texture mapping in computer graphics [24]. In both cases the realism of a synthetic scene is increased by insertion of snapshots of textures (either visual or aural) taken from actual objects and projected onto the model. 5.2 Cellular models A possible approach to simulation of complex dynamical systems is their decomposition into a multitude of interacting particles. The dynamics of each of these particles are discretized and quantized in some way to produce a finite-state automaton (a cell), suitable for implementation on a processing element of a parallel computer. The discrete dynamical system consisting of a regular lattice of elementary cells is called a cellular automaton [25], [26]. The state of any cell is updated by a transition rule which is applied to the previous-step state of its neighborhood. When the cellular automaton comes from the discretization of a homogeneous and isotropic medium it is natural to assume functional homogeneity and isotropy, i.e. all the cells behave according to the same rules and are connected to all their immediate neighbors in the same way [25]. If the cellular automaton has to be representative of a physical system, the state of cells must be characterized by values of selected physical parameters, e.g. displacement, velocity, force. Several approaches to physically-based sound modeling can be recast in terms of cellular automata, the most notable being the CORDIS-ANIMA system introduced by Cadoz and his co-workers [27], [28], [29], who came up with cells as discrete-time models of small mass-spring-damper systems, with the possible introduction of nonlinearities. The main goal of the CORDIS-ANIMA project was to achieve high degrees of modularity and parallelism, and to provide a unified formalism for rigid and flexible bodies. The technique is very expensive for an accurate sequential simulation of wide vibrating objects, but is probably the only effective way in the case of a multiplicity of micro-objects (e.g. sand grains) or for very irregular media, since it allows an embedding of the material characteristics (viscosity, etc.). An example of CORDIS- ANIMA network discretizing a membrane is shown in figure??, where we have surrounded by triangles the equal cells which provide output variables depending on the internal state and on input variables from neighboring cells. Even though the CORDIS-ANIMA system uses heterogeneous elements such as matter points or visco-elastic links, a network can be restated in terms of a cellular automaton showing functional homogeneity and isotropy. A cellular automaton is inherently parallel, and its implementation on a parallel computer shows excellent scalability. Moreover, in the case of the multiplicity of micro-objects, it has shown good effectiveness for joint production of audio and video simulations [30]. It might be possible to show that a two-dimensional cellular automaton can implement the model of a membrane as it is expressed by a waveguide mesh. How- 9

10 ever, as we will see in sections 5.3 and 5.4, when the system to be modeled is the medium where waves propagate, the natural approach is to start from the wave equation and to discretize it or its solutions. In the fields of finite-difference methods or waveguide modeling, theoretical tools do exist for assessing the correctness of these discretizations. On the other hand, only qualitative criteria seem to be applicable to cellular automata in their general formulation. 5.3 Finite-difference models When modeling vibrations of real-world objects, it can be useful to consider them as rigid bodies connected by lumped, idealized elements (e.g. dashpots, springs, geometric constraints, etc.) or, alternatively, to treat them as flexible bodies where forces and matter are distributed over a continuous space (e.g. a string, a membrane, etc.). In the two cases the physical behavior can be represented by ordinary or partial differential equations, whose form can be learned from physics textbooks [31] and whose coefficient values can be obtained from physicists investigations or from direct measurements. These differential equations often give only a crude approximation of reality, as the objects being modeled are just too complicated. Moreover, as we try to solve the equations by numerical means, a further amount of approximation is added to the simulated behavior, so that the final result can be quite far from the real behavior. One of the most popular ways of solving differential equations is finite differencing, where a grid is constructed in the spatial and time variables, and derivatives are replaced by linear combinations of the values on this grid. Two are the main problems to be faced when designing a finite-difference scheme for a partial differential equation: numerical losses and numerical dispersion. There is a standard technique [32], [33] for evaluating the performance of a finite-difference scheme in contrasting these problems: the von Neumann analysis. It can be quickly explained on the simple case of the ideal string (or the ideal acoustic tube), whose wave equation is [34] 2 p(x, t) t 2 = c 2 2 p(x, t) x 2, (14) where c is the wave velocity of propagation, t and x are the time and space variables, and p is the string displacement (or acoustic pressure). By replacing the second derivatives by central second-order differences, the explicit updating scheme for the i-th spatial sample of displacement (or pressure) is: p(i, n + 1) = 2 ( 1 c2 t 2 x 2 ) p(i, n) p(i, n 1) + c2 t 2 2 [p(i + 1, n) + p(i 1, n)], (15) x where t and x are the time and space grid steps. The von Neumann analysis assumes that the equation parameters are locally constant and checks the time evolution of a spatial Fourier transform of (15). In this way a spectral amplification factor is found whose deviations from unit magnitude and linear phase give respectively the numerical loss (or amplification) and dispersion errors. For the scheme (15) it can be shown that a unit-magnitude amplification factor is ensured as long as the Courant-Friedrichs-Lewy condition [32] c t x 1 (16) is satisfied, and that no numerical dispersion is found if equality applies in (16). A first consequence of (16) is that only strings having length which is an integer number of c t are exactly simulated. Moreover, when the string deviates from ideality and higher spatial derivatives appear (physical dispersion), the simulation becomes always approximate. In these cases, the resort to implicit schemes can allow the tuning of the discrete algorithm to the amount of physical dispersion, in such a way that as many partials as possible are reproduced in the band of interest [35]. 10

11 It is worth noting that if c in equation (14) is a function of time and space, the finite difference method retains its validity because it is based on a local (in time and space) discretization of the wave equation. Another advantage of finite differencing over other modeling techniques is that the medium is accessible at all the points of the time-space grid, thus maximizing the possibilities of interaction with other objects. When the objects being simulated are rigid bodies, they can be described by ordinary differential equations ẏ(t) = F[y(t), u(t), t], (17) y(0) = y 0, being u(t) the vector describing the set of input signals and y 0 initial conditions. Numerical analysis developed a plethora of techniques for their integration [32] transforming Eq.(17) into difference equations y(n + 1) = F d [y(n), u(n), n]. (18) However, attention must be paid to stability issues and to the correct reproduction of important physical attributes. These issues are strongly dependent on the numerical integration technique and on the sampling rate which are to be used. In most of the cases, there is no better method than trying several techniques and comparing the results, but the task is often facilitated by the fact that the strong nonlinearities are lumped. For example, in [36], the dynamics of a clarinet reed is discretized by using a fourth-order Runge-Kutta method, Euler differencing, and bilinear transformation [37]. The Runge-Kutta method turns out to be unstable for low sampling rates, while Euler differencing shows a poor reproduction of the characteristic resonance of the reed, due to numerical losses. For that specific case, the best choice seems to be the bilinear transform, which corresponds to a trapezoidal integration of the differential equations, possibly with some warping of the frequency axis [38] for adjusting the resonance central frequency. The discretization by impulse invariance [37] is also a reliable tool when aliasing can be neglected, and its performance is often preferable to bilinear transformation in acoustic modeling because it is free of frequency warping and artificial damping. Other discretization methods have recently been compared using the clarinet reed as a testbed [39]. As a result, it seems that a technique that employs polynomial interpolation of the input signals [40] gives the best reproduction of the reed resonance at an affordable cost. This latter technique can be interpreted as an extension of the impulse invariance that includes some antialias filtering. Further studies are needed to establish the most suitable discretization techniques for the many kinds of lumped dynamics, with special attention to be paid to the looped connection of lumped non-linear elements with memoryless nonlinearities and distributed resonators. Several techniques from signal processing and numerical analysis are yet to be experimented, while some general methodologies are just being proposed. In this respect, section 5.4 will show how, switching to a wave-variable representation of the physical quantities, it is possible to apply the paradigms of Wave Digital filters and Waveguide networks to the lumped and distributed elements respectively. 5.4 Wave models When discretizing physical systems a key role is played by the efficiency and accuracy of the discretization technique. Namely, we would like to be able to simulate simple vibrating structures and exciters with no artifacts (e.g. aliasing, or non-computable dependencies) and with low computational complexity. Due to its good properties with respect to these two criteria, one of the most popular ways of approaching physical modeling of acoustic systems makes use of wave variables instead of absolute physical quantities. Given the dual physical variables p and u (let us call them pressure and velocity), the pressure waves are defined as p + = (p + Z 0 u)/2, p (19) = (p Z 0 u)/2, where Z 0 is an arbitrary reference impedance. 11

12 When wave variables are adopted in the digital domain for representing lumped components this approach is called Wave Digital Filtering [41] It is possible to show that a lumped component having impedance Z(z) = P (z)/u(z) can be represented in pressure waves by R(z) = P (z) P + (z) = Z(z) Z 0 Z(z) + Z 0. (20) The reference impedance Z 0 is chosen in such a way that there is at least one delay element in any signal path connecting p + with p. The complete wave network is derived by applying the Kirchhoff principles [42] to junctions of components derived by the previous steps (wave-scattering formulation of the network) and to abrupt changes of the characteristic impedance. On the other hand, when the components to be modeled are distributed wave-propagating media, Digital Waveguide Networks [43] can be used to simulate them. In these models the physical variables are decomposed into their respective wave variables and their propagation is simulated by means of delay lines. Low-pass and all-pass filters are added to simulate dissipative and dispersive effects in the medium. As opposed to finite differencing, which discretize the wave equation (see eqs. (14) and (15)), waveguide models come from discretization of the solution of the wave equation. The solution to the one-dimensional wave equation (14) was found by D Alembert in 1747 in terms of traveling waves: p(x, t) = p + (t x/c) + p (t + x/c). (21) Eq. (21) shows that the physical quantity p (e.g. string displacement or acoustic pressure) can be expressed as the sum of two wave quantities traveling in opposite directions. In waveguide models waves are sampled in space and time in such a way that equality holds in (16). If propagation along a one-dimensional medium, such as a cylinder, is ideal, i.e. linear, non-dissipative and non-dispersive, wave propagation is represented in the discrete-time domain by a couple of digital delay lines (Fig. 2), which propagates wave variables, as defined in (19) with Z 0 characteristic impedance of the medium. As a slight generalization, it can be seen that the wave equation in a cone is identical to the wave equation in a cylinder (eq. (14), except that p(x, t) is replaced by x p(x, t) where x is the radial position along the cone axis. Thus, the solution is a superposition of left- and right-going traveling wave components, scaled by 1/x and can still be implemented by a couple of delay lines. p + (t) Wave Delay p + (t - nt) p - (t) Wave Delay p (t - + nt) Figure 2: Wave propagation propagation in a ideal (i.e. linear, non-dissipative and non-dispersive) medium can be represented, in the discrete-time domain, by a couple of digital delay lines. Let us consider deviations from ideal propagation due to losses and dispersion in the resonator. Usually, these linear effects are lumped and simulated with a few filters which are cascaded with the delay lines. Losses due to terminations, internal frictions, etc., give rise to gentle low pass filters, whose parameters can be identified from measurements [23]. Wave dispersion, which is often due to medium stiffness, is simulated by means of allpass filters whose effect is to produce a frequency-dependent propagation velocity [44]. In order to increase the computational efficiency, delay lines and filters should be lumped into as few processing blocks as possible. However, when considering the interaction with an exciter or signal pick-up 12

13 from certain points of the resonator, the process of commuting and lumping linear blocks must be done with care. If the excitation is a velocity signal injected into a string, it will produce two velocity waves outgoing from the excitation point, and therefore at least two delay lines will be needed to represent propagation. The process of commuting and lumping must maintain the semantics at the observation points, while in the other points of the structure it is not necessary to have a strict correspondence with the physical reality. Another aspect that we like to mention is that of simulating fractional delays. This is necessary when modeling musical instruments, since the proper tuning usually requires a space discretization much finer than dictated by the sample rate. More generally, fractional delay lengths are needed whenever time-varying acoustic objects (such as a string which is varying its length) are being modeled by digital waveguide networks. For this purpose, allpass filters or Lagrange interpolators of various orders can be used [45], the former suffering from phase distortion in high frequency, the latter suffering from both phase and amplitude distortion. However, low-order filters of both families can be used satisfactorily in most practical cases. In other cases, the problem of designing a tuning filter is superseded by the more general problem of modeling wave dispersion [46]. When the physical medium is changing its internal properties during vibration (e.g. a string exhibiting tension modulation), we should use a digital delay line that allows a continuous variation of the spatial sampling rate [47], or integrate the effects of the internal changes along the whole resonator length so that it becomes possible to treat them in a lumped fashion as length modulations [48]. So far, we have talked about one-dimensional resonators, but many musical instruments (e.g. percussions) and most of the real-world objects are subject to deformation along several dimensions. The algorithms presented so far can be adapted to the case of multidimensional propagation of waves, even though new problems of efficiency and accuracy arise. All the models grow in computational complexity with the increase of dimensionality, and for any of them, the choice of the right discretization grid is critical. For example, a rectangular waveguide mesh can be effective for simulating a vibrating flexible membrane, but the simulation of wave propagation turns out to be exact only along the diagonals of the mesh, while elsewhere it is affected by a dispersive phenomenon due to the fact that we are simulating circular waves by portions of plane waves. Waveguide meshes are shown to be equivalent to special kinds of finite-difference schemes [49], so that the von Neumann analysis can be used to evaluate the numerical properties of the algorithms. Special attention has to be paid to the dependence of the dispersion factor on frequency, direction, and mesh geometry, because this influences the distribution of resonances, and therefore affects the tone color and intonation. For membrane simulation, one of the most accurate yet efficient meshes is the triangular mesh [50], [51]. Interpolated waveguide meshes have recently been introduced for improving the accuracy while using simple geometries [52]. For three-dimensional wave propagation, the tetrahedral mesh is very attractive because its junctions can be implemented without any multiplication [53]. As compared to more conventional numerical techniques, such as finite difference schemes, the waveguide meshes offer the advantage that all the signals in the discrete-time system have a physical meaning, so that it is relatively straightforward to augment the model with some extra load (e.g. the air load for a membrane) or with nonlinearities. 6 Non linear musical oscillators Nonlinearities assume a great importance in acoustic systems, especially where a wave-propagation medium is excited. A good model for these nonlinear mechanisms is essential for timbral quality, and is the real kernel of a physical model, which could otherwise be reduced to linear postprocessing of an excitation signal. Since the area where the excitation takes place is usually small, it makes sense to use lumped models for the excitation nonlinearity. In some cases, physical measurements provide a representation of the relation among some physical variables involved in excitation, and this relation can be directly implemented in the simulation. For example, for a simplified bowed string the transversal velocity as a function of force can be 13