Digital Sound Synthesis by Physical Modeling U sing the Functional Transformation Method

Transcription

1 Digital Sound Synthesis by Physical Modeling U sing the Functional Transformation Method

2 Digital Sound Synthesis by Physical Modeling U sing the Functional Transformation Method Lutz Trautmann and Rudolf Rabenstein Telecommunication Laboratory, LMS Erlangen, Germany Springer-Science+Business Media, LLC

3 Library of Congress Cataloging-in-Publication Data Trautmann, Lutz. Digital sound synthesis by physical modeling using the functional transformation method /Lutz Trautman and Rudolf Rabenstein. p. cm. Includes bibliographical references and index. ISBN ISBN (ebook) DOI / Frequency synthesizers. 2. Transformations (Mathematics) 3. Sound-Recording and reproducing-digital techniques-mathematics. 4. Vibration-Mathematical models. I. Rabenstein, Rudolf. 1I. Title. TK7872.F73T ' 486-dc ISBN Springer Science+Business Media NewYork Originally published by Kluwer Academic/Plenum Publishers, New York in 2003 Softcover reprint of the hardcover 1 st edition A C.I.P. record for this book is available from the Library of Congress All rights reserved No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanicai, photocopying, microfilming, recording, or otherwise, witbout written permission from tbe Publisher, with the exception of any material supplied specifically for the purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the work

4 Foreword This book considers signal processing and physical modeling methods for sound synthesis. Such methods are useful for example in music synthesizers, computer sound cards, and computer games. Physical modeling synthesis has been commercialized for the first time about 10 years ago. Recently, it has been one of the most active research topics in musical acoustics and computer music. The authors of this book, Dr. Lutz Trautmann and Dr. Rudolf Rabenstein, are active researchers and inventors in the field of sound synthesis. Together they have developed a new synthesis technique, called the functional transformation method, which can be used for producing musical sound in real time. Before this book, they have published over 20 papers on the topic in journals and conference proceedings. In this excellent textbook, the results are combined in a single volume. I believe that this will be considered an important step forward for the whole community. The functional transformation method is proposed as a new way of designing physically based synthesis models for musical instruments. The derivation of the method uses an elegant technique, the Sturm-Liouville transformation, which is rarely used in acoustic signal processing. The resulting signal processing structure for modeling linear systems is similar to that used in modal synthesis, that is, a parallel connection of second-order filters. However, the functional transformation method offers certain advantages over the modal synthesis technique: most importantly, it avoids the frequency errors, which occur due to discretization in the modal synthesis. The functional transformation method also allows nonlinear interconnections of structures, which is important for many musical systems. The computational cost of an implementation that uses a second-order r"sonator for each vibrating mode can be seen as a disadvantage. Fortuv

5 vi DIGITAL SOUND SYNTHESIS USING THE FTM nately, computers are very fast and are getting faster all the time. Even today, a polyphonic real-time synthesizer based on the functional transformation has been realized with a multi-processor system. It will be an interesting future research task to develop structures that decrease the computational load of the functional transformation method. However, as computers keep on getting faster, the computational load will not be an issue after some years. In addition to introducing the new physical modeling technique, this book gives a brief general and historical overview of the technical field of sound synthesis. Basic wavetable, granular, additive, subtractive, and FM synthesis techniques and some of their modifications are explained. The fundamental physics of musical instruments are also covered. Two musical structures, a vibrating string and a drum membrane, are discussed in detail, and the partial differential equations related to these systems are derived. Some former physical modeling methods are tackled in detail: the finite difference, the digital waveguide, and the modal synthesis methods. The theoretical basis of the modal synthesis method is elaborated with care, because it is closely related to the new method. Finally, this text compares the new method with the previous ones. The digital waveguide and the finite difference method simulate vibrations in the time domain. The functional transformation method essentially has a frequency-domain point of view, just like the modal synthesis. All these methods can be derived from the wave equation or its extensions, and are thus just different viewpoints to the same physical reality. This comparison shows clearly that the new method has properties different from those of earlier methods. It is superior in many aspects while it may be weaker in some others, which is typical to all methods. It can be said that the functional transformation method is truly a novel and interesting method for physical modeling of musirpj instruments. Espoo, Finland, April 6, 2003 Vesa Valimaki, professor of audio signal processing Helsinki University of Technology Department of Electrical and Communications Engineering Laboratory of Acoustics and Audio Signal Processing Espoo, Finland

6 List of Figures 3.1 Illustration of a spatially I-D initial-boundary-value problem Construction of a guitar String vibration filtered at the bridge positions Strings terminated by separated impedance functions Strings terminated by an impedance network Construction of a kettle drum Forces on a string segment for the derivation of longitudinal string vibrations Forces and torques on a string segment for the derivation of torsional vibrations Forces and bending moments on a string segment for the derivation of transversal string vibrations Bowing force for transversal ~~cl torsional excited strings Forces and bending moments on a rectangular membrane segment for the derivation of bending membrane vibrations Illustration of a spatially I-D initial-boundary-value problem discretized by FDM Dependencies of the new calculated grid point on previous grid points in the FDM simulating the transversal vibrating lossy string FDM simulation of a transversal vibrating lossy and dispersive guitar string. 72 vii

7 viii DIGITAL SOUND SYNTHESIS USING THE FTM 4.4 Arrangement of the staggered grid points for FDM simulations with CDA of the wave equation in vector form FDM simulation of a longitudinal vibrating guitar string with boundary conditions of third kind Basic DWG stringed instrument model Basic DWG string model Illustration of a spatially I-D initial-boundary-value problem simulated with the DWG DWG simulation of the guitar nylon 'B' string vibration D rectangular DWM with scattering junctions between both polarizations Analytically calculated frequencies and frequencies used by the MS for a longitudinal vibrating guitar string Realization of the basic MS algorithm General procedure of the FTM solving initial-boundary-value problems defined in form of PDEs Multidimensional transfer function model derived from scalar PDEs Basic structure of the FTM simulations derived from scalar PDEs Illustration of the spatially 1-D initial-boundaryvalue problem simulated with the FTM Multidimensional transfer function model derived from vector PDEs Basic structure of the FTM simulations derived from vector PDEs General procedure of the FTM to solve initialboundary-value problems with nonlinear excitation functions MD implicit equation derived from scalar PDEs with a nonlinear excitation force Basic structure of FTM simulations derived from scalar PDEs with a nonlinear excitation force lvid implicit equation derived from scalar PDEs with solution-dependent coefficients Basic structure of FTM simulations derived from scalar PDEs with solution-dependent coefficients. 143

8 Figures ix 5.12 FTM simulation of a transversal vibrating guitar string FTM simulation of a longitudinal vibrating guitar string FTM simulation of a longitudinal vibrating guitar string with boundary conditions of third kind General procedure of the FTM to solve coupled initial-boundary-value problems given in form of two vector PDEs FTM simulation of two interconnected longitudinal vibrating guitar strings Discrete system of a struck string with a piano hammer FTM simulation of a hammer-string interaction in a piano Recursive system realization of one mode of the transversal vibrating string FTM simulation of a slapped bass string Discrete system of a plucked string with inherent tension-modulated nonlinearities FTM simulation of a tension-modulated vibrating string FTM simulation of the tension-modulated vibrating string with slap force FTM simulation of reverberation plate vibrations FTM simulation of circular drum head vibrations Dimensions of a simplified cuboid piano body FTM simulation of impulse responses within a resonant body of an upright piano. 6.1 Magnitude spectra of the simulated nylon 'B' gui tar string Combination of the FTM ami the DWG Magnitude response of the loss filter for a DWG model Phase delay of the dispersion filter Deflection and spectrum of the example nylon guitar string, simulated with the FTM and with the DWG

9 List of Tables 3.1 Comparison of three simplifying models for the subdivision of stringed instruments Parameters and variables used for the derivation of PDEs describing longitudinal, torsional and transversal string vibrations Parameters used for the derivation of PDEs describing bending membrane vibrations Parameters used for the derivation of PDEs describing resonant body vibrations Coefficients of the different initial-boundary-value problems in the unified scalar notation Physical parameters of a typical nylon 'B' guitar string Physical parameters used for the simulation of the hammer-string interaction in a piano and the fretstring interaction in a slapped bass Physical parameters used for the simulation of the vibrating quadratic reverberation plate and the circular drum head of a kettle drum Summary of the computational complexities of different systems simulated with FTM Computational cost of the DWG and the FTM simulations for a nylon 'B' guitar string simulated with Is = 44.1 khz. 199 xi

10 List of symbols Operators x y s * t * 0-1 (. )d OH OT 0* \7 fb,b, gb,b, f - b,b, gb,b fb,s,b {}, gb,s,b {} fi ii,s {} Dt DXl DO Im{} LNLO LwO Lw,s{} O(T) ReO W{} WDO scalar product between x and y convolution with respect to the temporal frequency variable s convolution with respect to time inverse operation of (.) discretized function of (.) hermitian of (.) transposed of (.) conjugate complex of (.) gradient or divergence operation boundary operators for vector PDEs adjoint boundary operators for vector PDEs boundary operators for scalar PDEs initial operator for vector PDEs initial operator for scalar PDEs first-order temporal differentiation first-order spatial differentiation temporal differential operator imaginary part of a complex function nonlinear scalar spatial differential operator scalar spatial differential operator self-adjoint scalar spatial differential operator higher order terms (depending linearly on T) real part of a complex function scalar differential operator containing mixed derivatives part of W {} containing temporal derivatives xiii

11 xiv Wd} WL,sO Y(s) f}(/-l) ij( t) y'(j:l) LO t{} DIGITAL SOUND SYNTHESIS USING THE FTM part of W {} containing spatial derivatives part of W {} containing self-adjoint spatial derivatives y(t) in the temporal frequency domain y(x) in the spatial frequency domain first-order temporal derivative of y (t) first-order spatial derivative of y(xd matrix spatial operator for vector PDEs adjoint spatial operator for vector PDEs Transformations CO TO ZO Laplace transformation Sturm-Liouville transformation z-transformation Constants 7f e 9 [N] j On,1J1 In circle constant Euler number constant of gravity imaginary unit zero matrix of order n x m unity matrix of order n Functions Scalars 13f t (Jrt "til 0(-) 00(-) L 1( ) a/i a(} damp a(} a e1as (} au atp [l/s] [rad N/m2] [rad N/m2] [rad N/m2] [N/m2] [Njm 2 ] discrete eigenvalue function in f-l adjoint discrete eigenvalue function in /-L discrete eigenvalue function in /-L continuous delta impulse discrete delta impulse discrete step function decay rate associated to the f-l-th mode rotational strain on a string segment dissipative rotational strain on a string segment elastic rotational strain on a string segment longitudinal strain on a string segment bending strain on a string segment

12 Symbols damp a<p a elas <p Tp CPAS cp wlj W~l WAS anl, bnl by dd(s) eloss fa fe fel fe2 J,lin el J,NL el fy fy,b fy,t fd fnl hf(t) damp m B m elas B mb damp m T mel as T mt nas(t) p~ 1 VH 'US V X1 [N/m2] [N/m2] [s] [rad] [rad] [rad/s] [rad/s] [rad/s] [rad/m] [N] [N] [N/m] [N/m2] [N/m] [N/m] [N] [N] [N] [N] [Nm] [N m] [N m] [Nm] [Nm] [Nm] [N/m2] [m/s] [m/s] [m/s] dissipative bending strain on a string segment elastic bending strain on a string segment normalized phase delay phase offset in additive synthesis bending angle of transversal vibrating structures angular frequency of ratational string motion angular frequency associated to the pah mode angular temporal frequency in additive synthesis nonlinear functions bending of the vibrating structure finite polynomial in S adaptation error in loss filter design internal stress force of the vibrating structure excitation force on a segment of the vibrational structure excitation force density for strings excitation force density for membranes linear part of the excitation force density for strings nonlinear part of the excitation force density for strings internal restoring force on transversal vibrating strucutures internal bending force of transversal vibrating structures force on transversal vibrating structures caused by the applied tension laminar air flow force nonlinear function I-D time-dependent filter function dissipative bending moment elastic bending moment bending moment dissipative torsional moment elastic torsional moment torsional moment white or colored noise source in additive synthesis fluctuating air pressure displacement of a longitudinal vibrating string hammer velocity in the piano string surface velocity particle velocity in longitudinal vibrating strings xv

13 xvi Vy [m/s] VB [m/s] W Jl, (loss,disp) WD(S) Y YAS YH YH BAS DAP Eo, Evibr[Nm] Edef [Nm] [Nm] Etot G(e,i,b) (/-L, s) H(z) Hloss(Z) Hdisp(Z) Hfd(Z) IFM(t) ljl(-) K(/-L, x) NJl ZS<Il) ZB DIGITAL SOUND SYNTHESIS USING THE FTM particle velocity in transversal vibrations bow velocity weights in the loss filter and the dispersion filter design finite polynomial in S spatial deflection of the vibrating structure output signal of the additive synthesis method output signal of the string deflection filtered with HF piano hammer deflection amplitudes of sinusoids in additive synthesis frequency response of the dispersion filter vibrational energy of the string vibration deformation energy of the string vibration total energy of the string vibration MD transfer functions transfer function in the temporal frequency domain loss filter in the SDL of the DWG dispersion filter in the SDL of the DWG fractional delay filter in the SDL of the DWG modulation index in the FM Bessel function of order /-L scalar eigenfunction of the scalar SLT norm factor for the inverse SLT boundary impedance of string n boundary impedance of membranes Vectors B 'P2 did(s) diw(s) fy V V3 Y Yi Yi,H [grad] [N] [m/s] boundary function on boundary B vector of bending angles in membrane vibrations vector containing scalar polynomials in S vector containing scalar polynomials in S internal elastic force vector for transversal membrane vibrations vector containing excitation functions particle velocity vector in 3-D structures vector containing different output variables vector containing initial functions vector containing the initial conditionsof the piano hammer

14 Symbols K K P fib xvii eigenfunction vector of the vector SLT adjoint eigenfunction vector of the vector SLT arbitrary function vector unit vector normal to the boundary B Matrices A B C matrix containing loss terms matrix containing terms with spatial derivatives matrix containing terms with temporal derivatives Variables EO E<p Eu rj e K, fl, 1-/. flt,flt V VM P Po P~ (J' W We Wm D.xl bn, Cn C Cij do d1 d2 d3 [rad/m] [rad/m] [m/m] [rad] 3 [kg/m ] [kg/mal [kg/mal [l/s] [rad/s] [rad/s] [rad/s] [m/s] [kg/(m 2 s)] [kg/(m 3 s)] [kg/s] [kg/(m s)] rotational strain of a string segment angular strain of a string segment longitudinal strain of a string segment distance from the neutral fiber twisting angle of string segment specific heat ratio vectors containing integer variables vectors containing a limited number of integers vector containing integer variables Poisson ratio mass density static air mass density fiuctuating air mass density decay rate angular temporal frequency carrier angular temporal frequency in the FM modulation angular temporal frequency in the FM width of a string segment different variables in the FTM wave velocity different constants in the FDM frequency independent damping coefficient for membrane vibrations frequency independent damping coefficient for string vibrations frequency dependent damping coefficient for membrane vibrations frequency dependent damping coefficient for

15 xviii Is gdc h k i io tr mh ni, mi Po l' s t Xn,s [lis] [kg] [N/m2] [lis] [s] Xl,H Xn XF A [m2] Ar [m2] B [m, m 2 ] E [N/m2] G [N/m2] GD [N s/m2] [m4] Is hvl [m 3 ] h [m4] Kdisp KT [m4] L N NAs NMs p Rn,(e,i,b) (f.l) SH [N/m PH ] T [s] [N/m] TM DIGITAL SOUND SYNTHESIS USING THE FTM string vibrations sampling frequency DC gain of the loss filter in DWG thickness of the membrane spatial dimension of the vibrating structure discrete integer time step length of the vibrating string length of the vibrating string at rest excitation length on the string hammer mass discrete integer space step of coordinate Xi static air pressure radius of the string's circular cross section frequency variable of the Laplace transformation continuous time variable spatial coordinates of a vibrational structure segment, n = {I, 2, 3} hammer position on the string continuous space coordinates, n = {I, 2, 3} evaluated string position, filtered with a body filter cross section area excitation area for membrane excitation boundary of the vibrating structure Young's modulus elastic modulus of regidity, shear modulus damping modulus of regidity, shear modulus moment of initertia for bending string vibrations moment of initertia for membranes moment of initertia for torsional string vibrations number of dispersion filters in DWG torsional form factor normalized delay in the SDL number of second-order resonators in the FTM number of added partials in additive synthesis number of spatial points and simulated modes in the MS filter order of the dispersion filter residuals of an expansion into partial fractions piano hammer stiffness coefficient temporal sampling interval surface tension on the membrane

16 Symbols xix [N] [m, m2, m 3 ] tension on the string definition range of the vibrating structure spatial sampling interval of coordinate :Ti vector of continuous spatial coordinates spatial excitation position spatial output positions

17 Abbreviations and Acronyms I-D 2-D 3-D AAC ADSR BDA BEM CD CDA DAC DFT DSP DWG DWM DWN FDM FDA FEM FIR FLOPS FM FTM IIR LMS LTI LTSI MD MIDI MP3 one-dimensional two-dimensional three-dimensional advanced audio coding attack-decay-sustain-release envelope backward difference approximation boundary element method compact disk central difference approximation digital to analog converter discrete Fourier transformation digital signal processor digital waveguide method digital waveguide mesh digital waveguide network finite difference method forward difference approximation finite element method finite impulse response floating point operations frequency modulation functional transformation method infinite impulse response least mean squares linear time-invariant linear time- and space-invariant multi dimensional musical instrument digital interface fvlpeg-l Layer 3 xxi

18 xxii MPEG MPOS MS ODE PC PDE SDL SL SLT STFT TFM WDF DIGITAL SOUND SYNTHESIS USING THE FTM moving picture experts group multiplications per output sample modal synthesis ordinary differential equation personal computer partial differential equation single delay line loop in the DWG Sturm-Liouville Sturm-Liouville transformation short time Fourier transformation transfer function model wave digital filter

19 Contents 1. INTRODUCTION 2. SOUND-BASED SYNTHESIS METHODS Wavetable synthesis 1.1 Looping 1.2 Pitch shifting Enveloping Filtering Granular synthesis 2.1 Asynchronous granular synthesis 2.2 Pitch-synchronQu>; granular synthesis Additive synthesis Subtractive synthesis FM synthesis Combinations of sound-based synthesis methods PHYSICAL DESCRIPTION OF MUSICAL INSTRUMENTS General notation 16 Subdivision of a musical instrument into vibration generators and a resonant body Division of stringed instruments into single strings and the resonant body 19 xxi i i

20 xxiv DIGITAL SOUND SYNTHESIS USING THE FTM Construction of stringed instruments Fixed strings filtered with the resonant body Strings terminated with independent impedances Strings terminated with an impedance network Division of a kettle drum into a membrane and the kettle Construction of drums Drum body simulation by modifying the physical parameters of the membrane Drum body simulation by room acoustic simulation with the membrane as vibrating boundary 27 3 Physical description of string vibrations Longitudinal string vibrations Torsional string vibrations 'fransversal string vibrations Basic linear model Nonlinear excitation functions Nonlinear PDE with solution-dependent coefficients 51 4 Physical description of membrane vibrations Bending membrane vibrations 52 5 Physical description of resonant bodies 57 6 Chapter summary CLASSICAL SYNTHESIS METHODS BASED ON PHYSICAL MODELS 63 1 Finite difference method FDM applied to scalar PDEs FDM applied to vector PDEs 73 2 Digital waveguide method Digital waveguides simulating string vibrations Digital waveguide meshes simulating membrane vibrations 83 3 Modal synthesis 86 4 Chapter summary FUNCTIONAL TRANSFORMATION METHOD 95 1 Fundamental principles of the FTM FTM applied to scalar PDEs Laplace transformation 101

21 Contents xxv Sturm-Liouville transformation Transfer function model Discretization of the MD TFM Inverse Sturm-Liouville transformation Inverse z-transformation FTM applied to vector PDEs Laplace transformation Sturm-Liouville transformation Transfer function model Discretization of the MD TFM Inverse Sturm-Liouville transformation Inverse z-transformation FTM applied to PDEs with nonlinear excitation functions FTM applied to PDEs with solution-dependent coefficients Stability and simulation accuracy of the FTM Section summary Application of the FTM to vibrating strings Transversal string vibrations described by a scalar PDE Longitudinal string vibrations described by vector PDEs Boundary conditions of second kind Boundary conditions of third kind Two interconnected strings Transversal string vibrations with nonlinear excitation functions Piano hammer excitation Slapped bass Transversal string vibrations with tensionmodulated nonlinearities Application of the FTM to vibrating membranes Rectangular reverberation plate Circular drum heads Application of the FTM to resonant bodies Chapter summary COMPARISON OF THE FTM WITH THE CLASSICAL PHYSICAL MODELING METHODS 189

22 xxvi DIGITAL SOUND SYNTHESIS USING THE FTM 1 Comparison of the FTM with the FD'M Comparison and combination of the FTM with the DWG Comparison of the FTM with the DWG Combination of the DWG with the FTM Designing the loss filter Designing the dispersion filter Designing the fractional delay filter Adjusting the excitation function Limits of the combination Comparison of the FTM with the MS Chapter conclusions SUMMARY, CONCLUSIONS, AND OUTLOOK 213 Index 225