Undergraduate Texts in Mathematics

Undergraduate Texts in Mathematics Editors S. Axler F.W. Gehring K.A. Ribet

Paul Cull Mary Flahive Robby Robson Difference Equations From Rabbits to Chaos With 16 Illustrations

Paul Cull Dept. Computer Science Dearborn Hall Oregon State University Corvallis, OR 97331 USA pc@cs.orst.edu Mary Flahive Dept. Mathematics Kidder Hall Oregon State University Corvallis, OR 97331 USA flahive@math.orst.edu Robby Robson Eduworks 3520 Northwest Hayes Ave. Corvallis, OR 97330 USA rrobson@eduworks. com Editorial Board S. Axler Mathematics Department San Francisco State University San Francisco, CA 94132 USA F.W. Gehring Mathematics Department East Hall University of Michigan Ann Arbor, MI 48109 USA K.A. Ribet Department of Mathematics University of California at Berkeley Berkeley, CA 94720-3840 USA Mathematics Subject Classification (2000): 39-01, 39Axx, 68Rxx, 11B37, 11B39 Library of Congress Cataloging-in-Publication Data Cull, Paul, 1943 Difference equations: from rabbits to chaos / Paul Cull, Mary Flahive, Robby Robson. p. cm. (Undergraduate texts in mathematics) Includes bibliographical references and index. ISBN 0-387-23234-6 (alk. paper) 1. Difference equations. I. Flahive, Mary E., 1948 II. Title. III. Series. QA431.C85 2004 515.625 dc22 2004058968 ISBN 0-387-23234-6 (softcover) ISBN 0-387-23233-8 (hard cover) Printed on acid-free paper. 2005 Springer Science+Business Media, Inc. All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher (Springer Science+Business Media, Inc., 233 Spring Street, New York, NY 10013, USA), except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden. The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights. Printed in the United States of America. (EB) 9 8 7 6 5 4 3 2 1 SPIN 10950852 (softcover) SPIN 10967645 (hardcover) springeronline.com

Preface Some years ago we noticed that various seemingly disparate fields were using similar models and techniques to solve similar problems. From mathematics to computer science to engineering to biology, various forms of difference equations were appearing in research papers and in textbooks, but with no common background, the same results were being independently derived over and over again. As we were noticing this, some mathematics curricula were being revised with discrete mathematics replacing calculus as the first college mathematics course. New discrete mathematics courses were created, and several superb textbooks appeared. In some fields a year of discrete mathematics actually replaced a year of calculus, while in other fields students took both discrete mathematics and calculus. With these changes, what happened to difference equations? Some texts in discrete mathematics ignored them. Others had a few examples of difference equations as applications of proof by induction. Still others devoted a chapter to difference equations, but only solved a few special cases and/or represented generating functions (also called Z-transforms) as the principal or only method for finding solutions. With this lack of common background, texts on algorithms, signal processing, and population biology were still forced to devote chapters to the difference equations used in their areas. Even students who took several of these courses had difficulty seeing that they were working with the same difference equations in different contexts. Many instructors had written notes to flesh out the coverage given in texts, but such notes were of necessity usually so terse that students were led to believe that difference equations were very complicated and hard to understand.

vi Preface With these problems in mind, we set out to write a book on difference equations that is accessible to undergraduates. As a text, it is meant for undergraduate majors in one of the mathematical sciences, presumably in their junior or senior year. We ve written it for the student who likes to compute and is comfortable with mathematical proof, but the book can be profitably read by students who approach the subject from either a computational or theoretical point of view. We wanted our text to have an algorithmic spirit. In this book, each chapter leads to techniques that can be applied by hand to small examples and also can be programmed for larger examples. In many cases we give explicit algorithms, which we decided to write in pseudocode rather than in a specific programming language for several reasons. First, it is easy to translate from pseudocode into any reasonable programming language. Second, there are many programming languages available, and translating from one language to another is often more difficult than translating from pseudocode. Third, we are not sure that programming these algorithms is worth the effort, because for almost all of our examples there are highquality implementations readily available on the Web. It probably makes more sense to use one of these programs rather than to cobble together a program that will be used only a few times and/or will be prone to problems when the input is not exactly in the form assumed by the programmer. A number of mathematically oriented computer packages are also available. For example, MATLAB, Maple, and Mathematica all have packages that will solve difference equations and recurrence relations. In many cases these packages give numeric answers as well as symbolic solutions when possible. Using these packages is much simpler than programming from scratch. In this book we start with the old story of Fibonacci s rabbits and progress through several generalizations, ending with some nonlinear difference equations. We deal with familiar mathematical structures such as the real numbers, the complex numbers, the integers, and the integers modulo an integer. We were tempted to discuss more general structures in order to show, for example, how theories of computation could be represented as difference equations, but we soon discovered that this would result in either a very large book or a very formal book, which would be at variance with our goal of accessibility. After developing the theory and techniques for solving linear difference equations in Chapters 2 to 4, we specialize to equations with nonnegative coefficients in Chapters 5 and 6 and then consider the generalization to matrix difference equations in Chapter 7. Chapter 8 considers equations over other rings, including integers modulo m and finite fields. Chapter 9 considers some issues in computational complexity, including divide-and-conquer algorithms. We end with some nonlinear systems in Chapter 10. Along the way we use linear algebra, develop formal power series, solve some combinatorial problems, visit Perron Frobinus theory, use graph theory, discuss pseudorandom number generation and integer factorization, and use the FFT to multiply polynomials quickly.

Computation vii There are four appendices serving different purposes. The first is a collection of worked examples, which are meant to supplement the early chapters of the book. Because the material in Appendices B and C is essential to an understanding of the book, we suggest working through them before beginning Chapter 2. Although many of the difference equations we consider have integer or real coefficients, it is often necessary to consider the coefficients as complex numbers. Appendix B gives the highlights of the complex analysis we use, and no prior experience is necessary to understand this appendix. On the other hand, only the most exceptional student could learn new material at the rate at which linear algebra is presented in Appendix C. One of the aims of this book is to show students that linear algebra is a powerful and coherent subject whose ideas have diverse applications, and we hope Appendix C is a helpful review. Appendix D outlines a method of Morris Marden [105] that can be used to decide when the general solution of a difference equation converges to zero. This appendix is not needed for an understanding of the book. Computation Most of our examples work with small difference equations, equations that can be completely solved by hand. In particular, for these equations their characteristic polynomials can be found, the roots of these polynomials can be computed exactly, and the associated eigenvector equations can be solved. While the theory we develop applies to both small and large equations, these computations may be difficult or impossible for large equations. For example, actually factoring polynomials is not possible in general, and rational computation of characteristic polynomials may require numbers with very many digits. Numerical approximation methods are often used for these computations, and we refer the interested reader to Acton [1], who gives a good introduction to numerical methods. (More serious users might refer to the compendium [131] or to the classic [170] by Wilkinson.) In general, we do not cover numerical methods. The one exception to this rule is our discussion of the use of Newton s method for finding the positive root of a nonnegative polynomial. We include this method for several reasons: it rapidly finds this root, the proof of its convergence and its speed of convergence are relatively easy, and the method is an example of a commonly encountered nonlinear difference equation.

viii Preface Notational Preliminaries In this book, we use the following fairly standard notation: Z is the set of integers. Z m is the set of integers modulo m. Z k is the set of all k-tuples with integer coordinates. Z k m is the set of all k-tuples of integers modulo m. N is the set of natural numbers, including 0; N = {0, 1, 2,...}. N + is the set of positive integers. Q is the set of rational numbers. R is the set of real numbers. F denotes a finite field. C is the set of complex numbers. R[x] isthesetofpolynomials with real coefficients. C[x] isthesetofpolynomials with complex coefficients. Z m [x] isthesetofpolynomials whose coefficients are integers modulo m. F[x] isthesetofpolynomials with coefficients from the finite field F. x is the floor of x R, the largest integer n with n x. x is the ceiling of x R, the smallest integer n with n x. k (mod m) means the equivalence class {k + jm : j Z }, while k mod m means the least nonnegative integer in the class k (mod m).

Contents Preface Computation............................. Notational Preliminaries....................... v vii viii 1 Fibonacci Numbers 1 1.1 The Rabbit Problem...................... 1 1.2 The Fibonacci Sequence.................... 2 1.2.1 Computing Fibonacci numbers............ 4 1.2.2 A formula for the Fibonacci numbers......... 5 1.2.3 Further Fibonacci facts................ 6 1.3 Notation for Asymptotic Analysis............... 6 1.4 Exercises............................ 7 2 Homogeneous Linear Recurrence Relations 11 2.1 The Solution Space of (HL).................. 12 2.2 The Matrix Form........................ 15 2.3 A Simpler Basis for the Solution Space............ 17 2.3.1 Distinct eigenvalues.................. 19 2.3.2 Repeated eigenvalues.................. 21 2.4 The Asymptotic Behavior of Solutions............ 25 2.5 Exercises............................ 28 3 Finite Difference Equations 33 3.1 Linear Difference Equations.................. 33

x Contents 3.1.1 First order equations................. 34 3.2 General and Particular Solutions............... 36 3.2.1 Finding a particular solution via summation..... 39 3.3 A Special Class of Linear Recurrences............ 41 3.4 Operator Notation....................... 45 3.5 The Shift Operator on the Space of Sequences........ 47 3.6 Formal Power Series...................... 50 3.6.1 Formal differentiation................. 55 3.6.2 An application of formal power series........ 56 3.7 Exercises............................ 58 4 Generating Functions 67 4.1 Counting Strings with Some Restrictions........... 67 4.2 An Overview of the Generating Function Technique.... 70 4.2.1 Rational representation................ 75 4.3 A Review of Partial Fractions................. 76 4.4 Examples of the Generating Function Technique...... 82 4.4.1 The Catalan numbers................. 83 4.4.2 Stirling numbers of the second kind......... 85 4.5 Reversion of Generating Functions.............. 87 4.5.1 Using the Fourier Transform............. 91 4.6 Exercises............................ 94 5 Nonnegative Difference Equations 101 5.1 Nonnegative Polynomials................... 102 5.1.1 The dominant root................... 102 5.2 When are integer solutions rounded powers of an eigenvalue? 106 5.2.1 Using the Rounding Theorem............. 110 5.3 Estimation of the Roots.................... 113 5.3.1 Estimation of the dominant root........... 113 5.3.2 Estimation of the second root............. 113 5.4 Calculation of the Roots.................... 116 5.4.1 The rate of convergence in Newton s method.... 121 5.5 Asymptotic Size of Solutions................. 125 5.5.1 Homogeneous nonnegative recurrences........ 125 5.5.2 Nonhomogeneous nonnegative equations....... 127 5.6 Exercises............................ 132 6 Leslie s Population Matrix Model 137 6.1 Leslie s Model.......................... 137 6.1.1 How to tell whether a Leslie matrix is primitive... 141 6.2 Leslie s Convergence Theorem................. 142 6.3 Imprimitive Leslie Matrices.................. 144 6.3.1 A simple example................... 144 6.3.2 A special case: Only one positive fertility rate.... 145

Contents xi 6.3.3 Asymptotically periodic Leslie matrices....... 145 6.4 Companion Matrices...................... 147 6.4.1 Matrices with repeated eigenvalues.......... 155 6.5 Nonnegative Companion Matrices............... 157 6.5.1 Periodic nonnegative companion matrices...... 159 6.6 Back to Leslie Matrices.................... 164 6.6.1 Periodic Leslie matrices................ 165 6.6.2 Averaging........................ 168 6.7 The Limiting Effect of L on Nonnegative Vectors...... 169 6.7.1 The period of the total population.......... 171 6.8 Afterword............................ 173 6.9 Exercises............................ 174 7 Matrix Difference Equations 179 7.1 Homogeneous Matrix Equations................ 179 7.2 Nonnegative Matrix Equations................ 186 7.2.1 Applications to Markov chains............ 187 7.3 Graphs and Matrices...................... 189 7.3.1 Next node representation............... 193 7.3.2 Comments on imprimitivity.............. 194 7.4 Algorithms for Primitivity................... 198 7.4.1 Algorithm I....................... 198 7.4.2 Algorithm II...................... 202 7.5 Matrix Difference Equations with Input........... 206 7.5.1 Reduction to one dimension.............. 207 7.5.2 Reduction to homogeneous form........... 211 7.6 Exercises............................ 212 8 Modular Recurrences 217 8.1 Periodicity............................ 218 8.1.1 Periodicity of linear modular recurrences....... 221 8.1.2 Fast modular computations.............. 224 8.2 Finite Fields.......................... 225 8.3 Periods of First Order Modular Recurrences......... 227 8.3.1 First order modular recurrences with maximal period 230 8.4 Periodic Second Order Modular Recurrences........ 232 8.4.1 Periods of modular Fibonacci sequences....... 233 8.5 Applications........................... 238 8.5.1 Application 1: Pseudorandom number generation.. 238 8.5.2 Application 2: Integer factorization.......... 242 8.6 Exercises............................ 246 9 Computational Complexity 253 9.1 Analysis of Algorithms..................... 254 9.1.1 Measuring run time.................. 254

xii Contents 9.1.2 An example: The Towers of Hanoi puzzle...... 256 9.2 Computer Arithmetic..................... 261 9.2.1 Addition and subtraction............... 262 9.2.2 Multiplication and division.............. 262 9.3 An Introduction to Divide-and-Conquer........... 263 9.3.1 Example: Polynomial multiplication......... 264 9.4 Simple Divide-and-Conquer Algorithms........... 268 9.4.1 Example 1: A return to polynomial multiplication. 270 9.4.2 Example 2: Matrix multiplication........... 271 9.4.3 Example 3: MERGESORT.............. 272 9.4.4 Example 4: Applications of Newton s method.... 273 9.5 The Fast Fourier Transform.................. 274 9.5.1 The general form of the Fast Fourier Transform... 276 9.5.2 The FFT when n =2 k................. 277 9.5.3 Fast evaluation and fast interpolation........ 280 9.5.4 The fast polynomial multiplication algorithm.... 281 9.6 Average Case Analysis..................... 284 9.6.1 The LARGETWO algorithm............. 284 9.6.2 The QUICKSORT algorithm............. 286 9.7 Exercises............................ 289 10 Some Nonlinear Recurrences 297 10.1 Some Examples......................... 297 10.2 Nonlinear Systems....................... 299 10.2.1 Sarkovskii s Theorem.................. 302 10.3 Chaos.............................. 303 10.3.1 A simple chaotic system................ 303 10.4 Local Stability......................... 307 10.4.1 Local stability of a fixed point............ 307 10.4.2 Local stability of a cycle................ 308 10.4.3 Local stability in two dimensions........... 310 10.5 Global Stability......................... 313 10.5.1 Staircase convergence................. 314 10.5.2 Nonmonotonic convergence.............. 315 10.6 Linear Fractional Recurrences................. 317 10.6.1 Asymptotic behavior.................. 318 10.6.2 Rational coefficients and periodicity......... 322 10.6.3 Chaotic-like behavior.................. 324 10.6.4 Invariant distributions................. 326 10.6.5 Proving global stability................ 330 10.6.6 Summary........................ 331 10.7 Conclusion........................... 332 10.8 Exercises............................ 333 A Worked Examples 337

Contents xiii A.1 All Simple Roots........................ 337 A.2 One Multiple Root....................... 342 A.3 One Multiple Root, Several Simple Roots.......... 345 A.4 The Input is γ1 n p 1(n) +γ2 n p 2(n)............... 346 B Complex Numbers 347 C Highlights of Linear Algebra 353 C.1 Vector Spaces and Subspaces................. 353 C.2 Linear Independence and Basis................ 354 C.3 Linear Transformations.................... 355 C.4 Eigenvectors........................... 356 C.5 Characteristic and Minimal Polynomials........... 358 C.6 Exercises............................ 359 D Roots in the Unit Circle 361 D.1 Marden s Method........................ 362 D.2 Exercises............................ 367 References 369 Index 381