Monographs in Theoretical Computer Science An EATCS Series

Transcription

1 Monographs in Theoretical Computer Science An EATCS Series Editors: W. Brauer J. Hromkovič G. Rozenberg A. Salomaa On behalf of the European Association for Theoretical Computer Science (EATCS) Advisory Board: G. Ausiello M. Broy C.S. Calude A. Condon D. Harel J. Hartmanis T. Henzinger T. Leighton M. Nivat C. Papadimitriou D. Scott

2 Manfred Droste Werner Kuich Heiko Vogler Editors Handbook of Weighted Automata

3 Editors Prof. Dr. Manfred Droste Universität Leipzig Institut für Informatik Leipzig, Germany Prof. Dr. Werner Kuich Technische Universität Wien Institut für Diskrete Mathematik und Geometrie 1040 Wien, Austria Prof. Dr.-Ing. Heiko Vogler Technische Universität Dresden Institut für Theoretische Informatik Dresden, Germany Series Editors Prof. Dr. Wilfried Brauer Institut für Informatik der TUM Boltzmannstr Garching, Germany brauer@informatik.tu-muenchen.de Prof. Dr. Grzegorz Rozenberg Leiden Institute of Advanced Computer Science University of Leiden Niels Bohrweg CA Leiden, The Netherlands rozenber@liacs.nl Prof. Dr. Juraj Hromkovič ETH Zentrum Department of Computer Science Swiss Federal Institute of Technology 8092 Zürich, Switzerland juraj.hromkovic@inf.ethz.ch Prof. Dr. Arto Salomaa Turku Centre of Computer Science Lemminkäisenkatu 14 A Turku, Finland asalomaa@utu.fi ISBN e-isbn Monographs in Theoretical Computer Science. An EATCS Series. ISSN Library of Congress Control Number: ACM Computing Classification (1998): F4.3, F1.1, F1.2 Mathematics Subjects Classification (2000): 68Q45, 68Q70, 03B50, 03B70, 03D05, 68R99, 68Q85 c 2009 Springer-Verlag Berlin Heidelberg This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilm or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer. Violations are liable to prosecution under the German Copyright Law. The use of general descriptive names, registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. Cover Design: KünkelLopka GmbH Printed on acid-free paper springer.com

4 Preface The purpose of this Handbook is to highlight both theory and applications of weighted automata. Weighted finite automata are classical nondeterministic finite automata in which the transitions carry weights. These weights may model, e.g., the cost involved when executing a transition, the amount of resources or time needed for this, or the probability or reliability of its successful execution. The behavior of weighted finite automata can then be considered as the function (suitably defined) associating with each word the weight of its execution. Clearly, weights can also be added to classical automata with infinite state sets like pushdown automata; this extension constitutes the general concept of weighted automata. To illustrate the diversity of weighted automata, let us consider the following scenarios. Assume that a quantitative system is modeled by a classical automaton in which the transitions carry as weights the amount of resources needed for their execution. Then the amount of resources needed for a path in this weighted automaton is obtained simply as the sum of the weights of its transitions. Given a word, we might be interested in the minimal amount of resources needed for its execution, i.e., for the successful paths realizing the given word. In this example, we could also replace the resources by profit and then be interested in the maximal profit realized, correspondingly, by a given word. Furthermore, if the transitions carry probabilities as weights, the reliability of a path can be formalized as the product of the probabilities of its transitions, and the reliability of a word could be defined again as the maximum of the reliabilities of its successful paths. As another example, we may obtain the multiplicity of a word, defined as the number of paths realizing it, as follows: let each transition have weight 1; for paths take again the product of the weights of its transitions (which equals 1); then the multiplicity of a word equals the sum of the weights of its successful paths. Finally, if in the latter example we replace sum by maximum, weight 1 is associated to a word if and only if it is accepted by the given classical automaton. v

5 vi Preface In all of these examples, the algebraic structure underlying the computations with the weights is that of a semiring. Therefore, we obtain a uniform and powerful automaton model if the weights are taken from an abstract semiring. Here the multiplication of the semiring is used for determining the weight of a path, and the weight of a word is then obtained by the sum of the weights of its successful paths. In particular, classical automata are obtained as weighted automata over the Boolean semiring. Many constructions and algorithms known from classical automata theory can be performed very generally for such weighted automata over large classes of semirings. For particular properties, sometimes additional assumptions on the underlying semiring are needed. Another dimension of diversity evolves by considering weighted automata over discrete structures other than finite words, e.g., infinite words, trees, traces, series-parallel posets, or pictures. Alternatively, in a weighted automaton, the state set needs not to be finite, so we can consider, e.g., weighted pushdown automata with states being pairs of states (in the usual meaning) and the contents of the pushdown tape. Moreover, weighted context-free grammars and algebraic systems arise from weighted automata over trees by using the well-known equivalence between frontier sets of recognizable tree languages and context-free languages. For the definition of weighted automata and their behaviors, matrices and formal power series are used. This makes it possible to use methods of linear algebra over semirings for more succinct, elegant, and convincing proofs. Weighted finite automata and weighted context-free grammars were first introduced in the seminal papers of Marcel-Paul Schützenberger (1961) and Noam Chomsky and Marcel-Paul Schützenberger (1963), respectively. These general models have found much interest in Computer Science due to their importance both in theory as well as in current practical applications. For instance, the theory of weighted finite automata and weighted context-free grammars was essential for the solution of classical automata theoretic problems like the decidability of the equivalence: of unambiguous context-free languages and regular languages; of deterministic finite multitape automata; and of deterministic pushdown automata. For the variety of theoretical results discovered, we refer the reader to the indispensable monographs by Samuel Eilenberg (1974), Arto Salomaa and Matti Soittola (1978), Wolfgang Wechler (1978), Jean Berstel and Christophe Reutenauer (1984), Werner Kuich and Arto Salomaa (1986), and Jacques Sakarovitch (2003). (See Chap. 1 for precise references.) On the other hand, weighted automata and weighted context-free grammars have been used as basic concepts in natural language processing and speech recognition, and recently, weighted automata have been used in algorithms for digital image compression.

6 Preface vii Since the publication of the mentioned monographs, the field of weighted automata has further developed both in depth and breadth. 1 The editors of this Handbook are very happy that international experts of the different areas agreed to write survey articles on the present shape of their respective field. The chapters of this Handbook were written such that a basic knowledge of automata and formal language theory suffices for their understanding. Next, we give a short overview of the contents of this Handbook. Part I provides foundations. More specifically, in Chap. 1, Manfred Droste and Werner Kuich present basic foundations for the theory of weighted automata, in particular, semirings, formal power series, and matrices. As is well known, regular and context-free languages can be obtained as least solutions of suitable fixed point equations. In Chap. 2, Zoltán Ésik provides an introduction to that part of the theory of fixed points that has applications to weighted automata and their behaviors, and to weighted context-free grammars in the shape of algebraic systems. Part II of this Handbook investigates different concepts of weighted recognizability. In Chap. 3, Zoltán Ésik and Werner Kuich develop the theory of finite automata starting from ideas based on linear algebra over semirings. In particular, they derive the fundamental Kleene Schützenberger characterization of the behaviors of weighted automata over Conway semirings. In Chap. 4, Jacques Sakarovitch presents the theory of rational and recognizable formal power series over arbitrary semirings and graded monoids. As a consequence, he derives that the equivalence of deterministic multitape transducers is decidable. A seminal theorem of J. Richard Büchi (1960) and Calvin C. Elgot (1961) shows the equivalence in expressive power between classical finite automata (over finite and infinite words) and monadic second-order logic. In Chap. 5, Manfred Droste and Paul Gastin present a weighted version of monadic second-order logic and derive corresponding equivalence results for weighted automata. In Chap. 6, Mehryar Mohri presents several fundamental algorithms for weighted graphs, weighted automata, and regulated transducers as, e.g., algorithms for shortest-distance computation, ε-removal, determinization, minimization, and composition. In Part III of this Handbook, alternative types of weighted automata and various discrete structures other than words are considered. In Chap. 7, Ion Petre and Arto Salomaa present the core aspects of the theory of algebraic power series in noncommuting variables, weighted pushdown automata, and their relationship to formal languages. In Chap. 8, Juha Honkala extends the theory of algebraic power series by considering Lindenmayerian algebraic systems and several restricted such systems. The following two chapters consider weighted automata acting on extensions of finite words. In Chap. 9, Zoltán Fülöp and Heiko Vogler survey the theory of weighted tree automata and weighted tree transducers. This combines classical results of weighted au- 1 For instance, see the biennial workshops on Weighted Automata: Theory and Applications (WATA) since 2002.

7 viii Preface tomata and transducers on words and of unweighted tree automata and tree transducers. In Chap. 10, Ina Fichtner, Dietrich Kuske, and Ingmar Meinecke present different weighted automata models for concurrent processes, formalized by traces and series-parallel posets, and analyze their relationships. They also consider two-dimensional extensions of words, namely pictures. Part IV deals with applications of weighted automata. In Chap. 11, Jürgen Albert and Jarkko Kari present the use of weighted automata and transducers for digital image compression and give comparisons with the image compression standard JPEG. In Chap. 12, George Rahonis describes the theory of fuzzy recognizable languages. This theory arises by considering weighted automata over particular semirings, namely bounded distributive lattices. In Chap. 13, Christel Baier, Marcus Größer, and Frank Ciesinski present the main concepts of Markov decision processes as an operational model for probabilistic systems, and basic steps for the (qualitative and quantitative) analysis against linear-time properties. In Chap. 14, Kevin Knight and Jonathan May address the reawakened interest in string and tree automata among computational linguists. The chapter surveys tasks occurring in natural language processing and shows their solutions by using weighted automata. Some of the chapters contain open problems. We hope that this will stimulate further research. Finally, we would like to express our thanks to all authors of this Handbook and to the referees for their careful work. Moreover, warm thanks go to Carmen Heger for her support in the technical compilation of the chapters. Manfred Droste Werner Kuich Heiko Vogler Leipzig Wien Dresden May 13, 2009

8 List of Contributors Jürgen Albert Universität Würzburg Informatik II Würzburg, Germany uni-wuerzburg.de Christel Baier Technische Universität Dresden Faculty of Computer Science Dresden, Germany Frank Ciesinski Technische Universität Dresden Faculty of Computer Science Dresden, Germany tu-dresden.de Manfred Droste Universität Leipzig Institut für Informatik Leipzig, Germany uni-leipzig.de Zoltán Ésik University of Szeged, Szeged, Hungary, and Rovira i Virgili University Tarragona, Spain Ina Fichtner Universität Leipzig Institut für Informatik Leipzig, Germany fichtner@informatik. uni-leipzig.de Zoltán Fülöp University of Szeged Department of Computer Science 6720 Szeged, Hungary fulop@inf.u-szeged.hu Paul Gastin LSV, ENS Cachan, INRIA, CNRS Cachan, France Paul.Gastin@lsv.ens-cachan.fr Marcus Größer Technische Universität Dresden Faculty of Computer Science Dresden, Germany groesser@tcs.inf.tu-dresden.de Juha Honkala University of Turku Department of Mathematics Turku, Finland juha.honkala@utu.fi ix

9 x List of Contributors Jarkko Kari University of Turku Department of Mathematics Turku, Finland Kevin Knight USC Information Sciences Institute Marina del Rey, CA USA Werner Kuich Institut für Diskrete Mathematik und Geometrie Technische Universität Wien 1040 Wien, Austria Dietrich Kuske Universität Leipzig Institut für Informatik Leipzig, Germany uni-leipzig.de Jonathan May USC Information Sciences Institute Marina del Rey, CA USA Ingmar Meinecke Universität Leipzig Institut für Informatik Leipzig, Germany uni-leipzig.de Mehryar Mohri Courant Institute of Mathematical Sciences New York, NY Google Research New York, NY USA Ion Petre Academy of Finland and Åbo Akademi University Turku 20520, Finland George Rahonis Aristotle University of Thessaloniki Department of Mathematics Thessaloniki, Greece Jacques Sakarovitch CNRS and Ecole Nationale Supérieure des Télécommunications Paris Cedex 13, France Arto Salomaa Turku Centre for Computer Science Turku 20520, Finland Heiko Vogler Technische Universität Dresden Faculty of Computer Science Dresden, Germany

10 Contents Part I Foundations Chapter 1: Semirings and Formal Power Series Manfred Droste and Werner Kuich Introduction MonoidsandSemirings FormalPowerSeries Matrices Cycle-FreeLinearEquations References Chapter 2: Fixed Point Theory Zoltán Ésik Introduction LeastFixedPoints ConwayTheories IterationTheories UniqueFixedPoints FixedPointsofLinearFunctions Inductive -Semirings CompleteSemirings IterativeSemirings FixedPointsofAffineFunctions CompleteSemiring SemimodulePairs Bi-inductive Semiring Semimodule Pairs References xi

11 xii Contents Part II Concepts of Weighted Recognizability Chapter 3: Finite Automata Zoltán Ésik and Werner Kuich Introduction FiniteAutomataoverSemirings FiniteAutomataoverArbitraryPowerSeriesSemirings FiniteAutomataoverConwaySemirings FiniteLinearSystems FiniteAutomataoverQuemirings Semiring SemimodulePairsandQuemirings FiniteAutomataoverQuemiringsandaKleeneTheorem FiniteLinearSystemsoverQuemirings References Chapter 4: Rational and Recognisable Power Series Jacques Sakarovitch Introduction RationalSeriesandWeightedRationalExpressions SeriesoveraGradedMonoid RationalSeries WeightedAutomata TheBehaviourofaWeightedAutomaton The Fundamental Theorem of Automata ConjugacyandCoveringofAutomata RecognisableSeriesandRepresentations TheFamilyofRecognisableSeries OtherProductsonRecognisableSeries SeriesonaProductofMonoids SeriesoveraFreeMonoid The Representability Theorem ReducedRepresentations ApplicationsoftheReductionofRepresentations Support of Rational Series Notes GeneralSources Notes to Sect. 2:RationalSeries Notes to Sect. 3:WeightedAutomata Notes to Sect. 4:RecognisableSeries Notes to Sect. 5:SeriesoveraFreeMonoid Notes to Sect. 6: Support of Rational Series References...171

12 Contents xiii Chapter 5: Weighted Automata and Weighted Logics Manfred Droste and Paul Gastin Introduction MSO-Logic and Weighted Automata Weighted Logics UnambiguousFormulas Definability Equals Recognizability Locally Finite Semirings Weighted Automata on Infinite Words Weighted Logics on Infinite Words ConclusionsandOpenProblems References Chapter 6: Weighted Automata Algorithms Mehryar Mohri Introduction Preliminaries Semirings WeightedTransducersandAutomata Shortest-Distance Algorithms All-PairsShortest-DistanceProblems Single-SourceShortest-DistanceProblems RationalOperations ElementaryUnaryOperations Fundamental Binary Operations Composition Intersection Difference Optimization Algorithms Epsilon-Removal Determinization WeightPushing Minimization Synchronization Conclusion References Part III Weighted Discrete Structures Chapter 7: Algebraic Systems and Pushdown Automata Ion Petre and Arto Salomaa Introduction Auxiliary Notions and Results AlgebraicPowerSeries...261

13 xiv Contents 3.1 DefinitionandBasicReductions InterconnectionswithContext-FreeGrammars NormalForms Theorems of Shamir and Chomsky Schützenberger Transductions PushdownAutomata PushdownTransitionMatrices S Σ -PushdownAutomata EquivalencewithAlgebraicSystems OtherTopics References Chapter 8: Lindenmayer Systems Juha Honkala Introduction IteratedMorphismsandRationalSeries LindenmayerianAlgebraicSeries D0LPowerSeries Other Power Series Generalizations of L Systems References Chapter 9: Weighted Tree Automata and Tree Transducers Zoltán Fülöp and Heiko Vogler Introduction Preliminaries GeneralNotation Trees AlgebraicConcepts TreeSeries WeightedTreeAutomata Bottom-upTreeAutomata RecognizableTreeSeries Closure Properties Support of Recognizable Tree Series DeterminizationofWeightedTreeAutomata Pumping Lemmata and Decidability Finite Algebraic Characterizations of Recognizable Tree Series EquationalTreeSeries RationalTreeSeries MSO-Definable Tree Series Other Models Related to Recognizable Tree Series Further Results IO-Substitution and Tree Series Transformations WeightedTreeTransducers TreeTransducers...364

14 Contents xv 5.2 TheBasicModel RestrictedModels CompositionandDecomposition The Inclusion Diagram of Some Fundamental wtt Classes Hierarchies FurtherModelsofWeightedTreeTransducers FurtherResults References Chapter 10: Traces, Series-Parallel Posets, and Pictures: A Weighted Study Ina Fichtner, Dietrich Kuske, and Ingmar Meinecke Introduction Traces WeightedDistributedSystems Other Formalisms: Presentations, Expressions, and Logics Relating the Formalisms HistoryandOverview Series-Parallel Posets Series-Parallel Posets and Bisemirings WeightedBranchingAutomata Rationality TheHadamardProduct HistoryandOverview Pictures PicturesandWeightedPictureAutomata Other Formalisms: Expressions and Logics Relating the Formalisms Decidability Issues HistoryandOverview References Part IV Applications Chapter 11: Digital Image Compression Jürgen Albert and Jarkko Kari Introduction ImageTypes WeightedFiniteAutomataandMulti-resolutionImages DrawingWFAImages An Encoding Algorithm PracticalImageCompressionUsingWFA WeightedFiniteTransducers(WFT) ParametricWeightedFiniteAutomata(PWFA)...472

15 xvi Contents 9 ConclusionsandOpenProblems References Chapter 12: Fuzzy Languages George Rahonis Introduction LatticesandFuzzyLanguages Fuzzy Recognizability over Bounded Distributive Lattices Fuzzy Recognizability over Finite Words Fuzzy Recognizability over Infinite Words Multi-valued MSO Logic FuzzyLanguages:AnOverview Fuzzy Languages over l-monoids FuzzyLanguagesoverResiduatedLattices FuzzyAutomatawithOutputs Fuzzy Abstract Families of Languages FuzzyTreeLanguages Applications References Chapter 13: Model Checking Linear-Time Properties of Probabilistic Systems Christel Baier, Marcus Größer, and Frank Ciesinski Introduction MarkovDecisionProcesses Maximal Reachability Probabilities Model Checking ω-regular Properties PartialOrderReduction Partially Observable MDPs Conclusion Appendix References Chapter 14: Applications of Weighted Automata in Natural Language Processing Kevin Knight and Jonathan May Background WFSTTechniquesforNaturalLanguageProcessing Example1:Transliteration Example2:Translation LanguageModeling ApplicationsofWeightedStringAutomata LanguageTranslation SpeechRecognition LexicalProcessing Tagging

16 Contents xvii 3.5 Summarization OpticalCharacterRecognition ApplicationsofWeightedTreeAutomata OpenProblems Conclusion References Index...597