Rhythm Tree Rewriting

Size: px

Start display at page:

Download "Rhythm Tree Rewriting"

Ethelbert Elliott
5 years ago
Views:

1 Rhythm Tree Rewriting Jean Bresson (Ircam) Pierre Donat-Bouillud (ENS Cachan/Rennes) Florent Jacquemard (INRIA, Ircam) IFIP WG1.6 July 13, 014

2 general perspective music symbolic representations in particular traditional western music notation processing of tree structured representations! general long term goal: symbolic MIR problems querying bases of music scores transformations, melodic similarity detection, genre classification, recommendation, detection of repetitions, automatic segmentation, musicological analysis! one particular target application: rhythm transcription

3 target application rhythm transcription: automatic generation of notation plan of the talk context: computer assisted (Music) composition problem: rhythm transcription data structure: rhythm trees term rewriting approach weighted tree automata

4 1 Assisted Composition hardware and software tools for authoring music, production of music scores

5 music notation editors score printing import/export MIDI and MusicXML Finale (MakeMusic) Sibelius (Avid)

6 algorithmic composition environments solving musical problems visual programming languages based on Lisp PWGL (Sibelius Academy) OpenMusic (Ircam)

7 Automatic Music Transcription

8 automatic music transcription: goal acoustical recording (audio file) (71,0.0)(74,1.18)(46,1.73)(5,.09)(5 symbolic timed trace (MIDI file, piano roll) notation (MusicXML )

9 automatic music transcription: tasks pitch tracking acoustical recording (audio file) audio MIR onset detection beat tracking symbolic timed trace (MIDI file, piano roll) symbolic MIR tempo/metric extraction structural segmentation notation (MusicXML )

10 automatic music transcription: applications recorded performance acoustical recording (audio file) MIDI keyboard (score edition) model of performance = score + deviations symbolic timed trace (MIDI file) algorithmic composition (OpenMusic ) agnostic notation (MusicXML ) this work, with focus on rhythm quantization

11 rhythm quantization (symbolic) 1. segmentation of the input timed trace each segment with constant tempo or known acceleration. identification of tempo / beat positions 3. local quantization on each segment (on-the-beat quantization) input: one segment = timed trace: sequence s of pairs (onset,duration) in R (ms) output: sequence t of pairs (onset, kind), with onsets in D, small discrete set (admissible subdivisions of beat) kind = note or silence number of notes = s

12 rhythm quantization : measures of quality timed trace transcription 1: high precision, high complexity transcription : lower precision, low complexity

Quantization of musical time Music and Connectionnism, MIT

13 alignment to grid Longuet-Higgins Mental Processes: Studies in Cognitive Science, 1987 Desain, Honing, de Rijk Quantization of musical time Music and Connectionnism, MIT Press depth 3 0 1/ 1 c grid of depth 3 and subdivision schema (3,, )

14 choice of the grid according to depth, number of divisions with user parameters or heuristics which grid for this? 3 4 which grid for that?

15 heuristics for grid selection Pressing, Lawrence Transcribe: A comprehensive autotranscription program ICMC 1993 Agon, Assayag, Fineberg, Rueda Kant: A critique of pure quantization ICMC 1994 Meudic OMquantify PhD Ircam, 005 given a predefined set T of template grids (user parameters) align the input segment s to grids of T select the best grid g T according to distance to s return alignement of s to g (converted to a score)

16 local quantization workflow OMquantify generation of several grids select best grid g (according to 3 distances) align input seq. s to g convert the alignement into OpenMusic Rhythm Tree return as score!! RT based approaches convert input seq. s into RT computations on RT(s)! return RT result as score

17 3 Rhythm Trees

18 rhythms and durations rhythmic values (fractions of the period) = symbolic notation tempo (frequency) durations (ms)

19 rhythm trees (RT) in traditional western music notation: durations are defined by recursive subdivisions of units

20 Lee The rhythmic interpretation of simple musical sequences Musical Structure and Cognition, 1985 rhythm CF grammars 4 3! < ! ˇ > ! ˇ (? S! < ! < ! ˇ > ! ˇ (?

21 OM rhythm trees (OMRT) trees are the standard data structure for OpenMusic Laurson Patchwork: A Visual Programming Language Helsinki: Sibelius Academy, 1996 Agon, Haddad, Assayag Representation et rendu de structures rythmiques JIM, labels in N = durations children = equal subdivisions of parent notes = leaves for numerical computations (not symbolic) list (4 (3 (1 ( 1)) 1 (1 (1 1 1))))

22 symbolic RT all sibling represent equal durations for symbolic computations (unary notation) terms over the following signature: inner nodes: labelled with arity = prime numbers -13 leaves: n : note r : rest s : slur. Sum with previous leaf in dfs ordering 1+ : add to next sibling G ˇ ˇ ˇ ˇ n n n n

23 symbolic RT (example ) G 44 ˇ` fl 3 fl ˇ ˇ ˇ ˇ fl 3 fl ˇ ˇ ˇ n 3 1+ n n 3 1+ n n n n n

24 symbolic RT (example 3) G 44 fl 3 fl ˇ ˇ ˇ. 1+ n 1+ 3 n n n

25 advantage of RT representation over string representation close to traditional music notation! keep the integrity constraint sum of durations = 1! groups of correlated events reflected in the tree structure = sequences of siblings preserved in local transformations

26 4 Local Quantization by RT Rewriting

27 principle 1. generate an initial tree t0 from input s with maximum precision, maximum complexity t0 complete closest to s given maximum depth, signature alignment to a complete grid!. simplify t0 into t using a set of rewrite rules of kinds: conservative rules (preserve durations) simplifying rules (do not preserve durations)! 3. return score corresponding to t

28 conservative rules merge-ns p(n, s,...,s)! n ˇ 8 ˇ! ˇ merge-r p(r,...,r)! r?! > merge-s p(s,...,s)! s 8 ˇ 8 ˇ! r 8 ˇ replace-s p(x 1,...,x m,t,1+,...,1+, n,y {z } 1,...,y n )! p(x 1,...,x m,t,n, s,...,s,y {z } 1,...,y n ) t = n or t = r or t = s, t = p(z 1,...,z p ) k k

29 ˇ ˇ ˇ conservative rules () 3 3/ 3(n, (s, n), s)! (n, n) ˇ 8 ˇ ˇ 8 ˇ! ˇ ˇ 5 5/ 5(n, s, (s, n), s, s)! (n, n) ˇ = ˇ ˇ ˇ = ˇ! ˇ ˇ 7/, 11/, 4/3 (n, 3(s, n, s)), (3(s, s, n), s)! 3(n, n, n) ˇ ˇ ˇ > ˇ ˇ ˇ 8 ˇ! ˇ ˇ ˇ 5/3 5(n, 3(s, s, n), s, 3(s, n, s), s)! 3(n, n, n) ˇ;ˇˇ ˇ = ˇ ˇ ˇ ; ˇ! ˇ ˇ ˇ

30 conservative rules (application) equational theory of rhythm notation the set of conservative rules is confluent and terminating! it can be used as a tool for simplifying rhythm notations identify equivalent rhythm notations

31 simplifying rules reduce inflate p(s, n,x 1,...,x m )! p(n, s,x 1,...,x m ) p(s, r,x 1,...,x m )! p(r, r,x 1,...,x m ) p(s,...,s, r)! p(s,...,s) {z } {z } m m+1 p 1 p(s,...,s, n),p {z } 0 (s,x 1...,x k )! p 1 p(s,...,s),p {z } 0 (n,x 1...,x k ) m m+1 p p p 1 p 0 (s,x 1,...,x`)! p 1 p 0 (n,x 1,...,x`) x p(s,...,s, n) {z } m x p(s,...,s) {z } m+1

32 TRS with regexp constraints there is an exponential number of simplifying rules currently implemented as LISP functions (1 function represents a family of rules) studying other compact rule-based representation, with tree, context and function variables regular constraints (variable in regular tree language)!!! Kutsia, Marin Matching with Regular Constraints, 005! problems: matching rewriting strategies (bottom-up) characterization of set of descendants

33 example initial tree (complete) 5 n s s s s s s s s r r r r r r r r r r r

34 example normalize (conservative) reduce (non-conservative) normalize (conservative) n s r r n s r r n s r r r r r s r r r

35 example () other initial tree (complete) n s s s s s s s s r r r r r r r r r r r

36 example normalize (conservative) inflate (non-conservative) normalize (conservative) r r n r n s s s s s s s s r n s s s s s s s s s

37 summary and questions 1. generate an initial tree t0 from input s several possibilities of t0. simplify t0 into t simplifying rules diverge (symbolic) exploration of space of reachable t choice of best t according to complexity and distance to s 3. return score corresponding to t" conversion of RT into notation (through OMRT)

38 5 Weighted Tree Automata

39 the set of initial trees, given an input sequence s! is a regular tree language Ls, recognized by a tree automaton As. exploration: the closure of Ls under rewriting is characterized by a tree automaton As *! constructed incrementally from As " and the rewrite rules (tree automata completion) choice of the best t: minimizing complexity and distance to s both computed by weighted tree automata

40 space exploration, selection best choice Bayesian approach to quantization Cemgil, Desain, Kappen Rhythm Quantization for Transcription Computer Music Journal 4(), 000. Cemgil Bayesian Music Transcription PhD Radboud University of Nijmegen, 004 p(t s) = p(s t ). p(t ) dist(s, t) square of a weighted Euclidian distance! 1 score complexity (length of Shannon code in bits) ad hoc performance model s = t / v + ε v = tempo (constant) ε = expressive timing deviation

space exploration, selection of best choice weighted tree automaton (WTA) A: finite bottom-up tree automaton with a weight for each transition. associates a weight A(t) to each term t.

41 space exploration, selection of best choice weighted tree automaton (WTA) A: finite bottom-up tree automaton with a weight for each transition. associates a weight A(t) to each term t. choice of t for s minimizing dist(s, t ). cpty(t ) where = Aws * (t ). Ac(t ) Aws * (t ) = dist(s, t ) if t recognized by As * = otherwise (ongoing) construction follows the construction of As * from As" Ac is a weighted tree automaton characterizing the user preferred notations (independently of s). It is learned online by quantization of previous segments.

42 construction of Ac (user s style) for each segment s 1. enumerate the k first t candidates according to Aws * (t ).Ac(t ) (k is a user parameter). print these t in traditional music notation 3. the user chooses his preferences, Ac is updated accordingly using extension of RPNI algorithm to WTA (Adrien Maire)

43 example proposition 1: ˇ (? d = 0.05, low complexity 5 proposition : ˇ 7 ˇ??? d = 0.05, higher complexity if the user chooses proposition 1 for this segment then A is left unchanged if the user prefers proposition then A is updated: 5-uplets will get higher weights for next segments

44 enumeration there exists a WTA A s.t. for all t, A(t) = Aws * (t ).Ac(t )! apply k-best algorithm to the weighted hypergraph presentation of A Huang, Chiang Better k-best parsing Parsing 005 improves Knuth extension to hyper graphs of Dijkstra s shortest path algorithm

45 conclusion approach for local rhythm quantization based on RT, involving 1. term rewriting. weighted tree automata! perspectives joined tempo inference and local quantization application of rhythm trees techniques (automata, transformations) to other symbolic MIR problems

trees representations in music Lerdahl and Jackendoff A Generative theory of tonal music MIT Press, 1983 Gilbert, Conklin A probabilistic context- free grammar for melodic

46 trees representations in music Lerdahl and Jackendoff A Generative theory of tonal music MIT Press, 1983 Gilbert, Conklin A probabilistic context- free grammar for melodic reduction Artificial Intelligence and Music, IJCAI, 007 structuration of a piece in a time-span tree Johanna Högberg Generating music by means of tree transducers CIAA, 005

CS229 Project: Musical Alignment Discovery

S A A V S N N R R S CS229 Project: Musical Alignment iscovery Woodley Packard ecember 16, 2005 Introduction Logical representations of musical data are widely available in varying forms (for instance,