The Fast Fourier Transform

Transcription

1 The Fast Fourier Transform A Mathematical Perspective Todd D. Mateer

2 Contents Chapter 1. Introduction 3 Chapter 2. Polynomials Basic operations The Remainder Theorem and Synthetic Division Modular reduction of a polynomial Multipoint polynomial evaluation 22 Chapter 3. Complex numbers Number systems Complex arithmetic Polar Representation of Complex Numbers Primitive Roots of Unity Euler s Formula Rotation transformations 38 Chapter 4. FFT algorithms The Binary Reversal Function Classical radix-2 FFT Operation count of the classical radix-2 algorithm Twisted radix-2 FFT Classical radix-4 FFT Twisted radix-4 FFT Radix-8 FFT algorithms Split-radix FFT Other 2-adic FFT algorithms Properties of the DFT FFTs of real data 87 Chapter 5. Inverse FFT algorithms Lagrangian Interpolation Classical radix-2 inverse FFT Twisted radix-2 IFFT Other 2-adic inverse FFT algorithms The duality property 103 Chapter 6. Polynomial multiplication algorithms Classical Multiplication Karatsuba multiplication FFT-based multiplication 114 1

3 2 CONTENTS Chapter 7. The engineering perspective The Discrete Fourier Series A mathematician s perspective of the engineer s FFT The two types of engineering FFT algorithms Convolution 132 Chapter 8. FFT algorithms for other input sizes The Ternary Reversal Function Classical radix-3 FFT Twisted radix-3 FFT Radix-5 FFTs Radix-6 algorithms 158 Chapter 9. Additional topics Additive Fast Fourier Transforms Computer Algebra Algorithms Truncated Fast Fourier Transform Implementation of FFT Algorithms Applications of the FFT Concluding Remarks 177 Appendix A. The relationship between the Fourier Transform and the Discrete Fourier Transform 179 Appendix B. Residue Rings Background Definitions The set of residues is a ring The residue ring (D\m,, ) is isomorphic to the quotient ring (D/(m),, ) Examples Concluding Remarks 200 Appendix C. The convolution theorem 203 Appendix. Bibliography 207

4 CHAPTER 1 Introduction During the second half of the twentieth century, the Fast Fourier Transform (FFT) has become one of the most important techniques in Electrical Engineering. This statement is supported by the fact that over 2,000 papers have been published on the topic since the 1960 s [22] and that a list of over 75 applications of the FFT are given in [7]. But what is the Fast Fourier Transform? This is not an easy question to answer and the response you get depends upon who you ask. After some introductory definitions, we will attempt to provide an answer to this question. First, according to [DSP] a signal is defined as follows: Signal. A signal is any physical quantity that varies with time, space, or any other independent variable or variables. Those that have completed a high school mathematics curriculum may think that the definition of signal closely matches the definition of a function. The only difference seems to be that a function is a mathematical concept, whereas a signal represents something tangible in the real world. In this text, the terms signal and function will be used interchangably. Signals can be classified in many different ways. The first type of signal of interest in this introductory section is: Analog signal. An analog signal is a function that is defined for all inputs in a specified interval. A signal is said to be continuous if it can be drawn on a piece of paper without lifting one s pencil off of the paper. A signal is said to be bandlimited if all of the nonzero outputs of the function are restricted to a finite interval of input values. 3

5 4 1. INTRODUCTION EXAMPLE Let x a (t) be the analog signal shown in the figure below. It consists of two triangles, each of which has a width of 40 milliseconds. The signal is zero for all inputs less than 20 milliseconds and for all inputs more than 140 milliseconds, so it is bandlimited. The signal can be drawn without lifting one s pencil off of the paper, so it is continuous. x a (t) ms t 160ms 1 Digital signal. A digital signal is a function that is only defined at certain values of time (usually multiples of an integer) and is a function that only has a finite set of possible values. Given an analog function x(t), a digital signal x(τ) can be constructed by sampling x(t) at evenly-spaced inputs determined by the sampling interval. EXAMPLE We are going to construct the digital signal x(τ) by sampling x a (t) above over the input range 0 ms t < 160 ms using 16 samples. To do so, the required sampling interval is 160 milliseconds 16 = 10 milliseconds. The resulting function is given below. x(τ) τ.5 1

6 1. INTRODUCTION 5 A true digital signal only has a finite number of possible outputs, so x(t) is typically rounded to the nearest finite value in this case. In this text, we will relax this assumption and work with discrete-time signals which sample an analog signal at evenly-spaced intervals, but place no restriction on the allowable output values. Now that we have seen the difference between an analog and discrete-time signal, we are ready to return to the goal of figuring out what is meant by the phrase Fast Fourier Transform. First, we will introduce the Fourier Transform which operates on analog signals. Next, we will proceed to the Discrete Fourier Transform which operates on discrete-time signals. Finally, we will explain what a Fast Fourier Transform is and how it relates to the Discrete Fourier Transform. In a typical signals analysis course in an Electrical Engineering curriculum, the Fourier transform of some analog signal f(t) is usually defined by the following formula F(s) = f(t) e 2πt s I dt. Here I is a symbol that represents 1, used with complex numbers, t is a real variable, and f(t) is a function that may produce either a real or a complex number output. For a person studying the FFT for the first time, this formula might be somewhat scary as it involves both Calculus and complex numbers. But the reader should take comfort in the fact the optional appendix is the only part of this book that will require any knowledge of Calculus and a full chapter of the book will be devoted to basic properties of complex numbers. The Discrete Fourier Transform of some discrete-time signal f(τ) is usually computed using either the formula or F(k) = F(k) = 1 N N 1 τ=0 N 1 τ=0 2π k/n τ I f(τ) e 2π k/n τ I f(τ) e depending on what book one picks up. The mathematical notation used in these formulas means to add up the expression that follows the Σ symbol at each value of τ from 0 to N 1. For example, if N = 4, then N 1 τ=0 τ = = 6

7 6 1. INTRODUCTION A question that is often asked at this point is how the Fourier transform and the discrete Fourier transform are related. This is also not an easy question to answer and requires a knowledge of Calculus. A fairly detailed discussion of this topic can be found in the appendix. For now, let it suffice to say that f(τ) is a function that samples f(t) uniformly at N locations. As N increases, f(τ) becomes a better and better approximation to f(t) and the Discrete Fourier Transform becomes a better and better approximation to the Fourier Transform. From this point forward, we will use the simpler notation f(τ) to represent the sampled version of f(t). To illustrate how the Fourier Transform and Discrete Fourier Transform relate to one another, let us consider what is probably the most popular example used in courses that cover the Fourier Transform. The so-called rectangle function is a function that takes on a value of 1 in the interval 1/2 < t < 1/2 and is defined to be zero elsewhere. 1 A graph of the rectangle function is given by 1 0 τ 1 One nice feature about this function is that it is symmetric about the vertical axis. This means that the graph to the left of the vertical line above is the mirror image of the graph to the right of the vertical line. When a function is symmetric about the vertical axis, the Fourier Transform definition simplifies to F(s) = 0 2 f(t) cos(2πt s) dt which eliminates the complex variables in this case. Earlier, it was mentioned that Calculus is not required to understand the material in this book. However, for the benefit of those readers that do have a background in Calculus, the Fourier transform of the rectangle function can be computed as follows: 1 At the values of t = 1/2 and t = 1/2 where the function suddenly jumps between 0 and 1, the rectangle function is usually defined to have a value of 1/2.

8 1. INTRODUCTION 7 F(s) = = 1/ cos(2π t s) dt sin(2 π t s) π s = sin(πs) πs which is defined for all values of s 0. The sinc function is defined using the above formula with the added condition that sinc(0) = 1. Two perspectives of the sinc function are given below. The domain of this function is all of the real numbers, so it is impossible to show a graph of the entire function. Observe that the sinc function has a value of zero for all integer inputs with the exception that the function has an output of one when the input is zero. This is a desirable feature of the sinc function which is useful in engineering. ] 1/2 t= τ 0 τ 1 1 To compute the Discrete Fourier Transform of the rectangle function, we sample the above function at N evenly spaced points over an interval of size T 0. To simplify the notation, we will use the interval T 0 /2 x < T 0 /2. The parameter T 0 can be chosen to be whatever value that one wishes. Here, we will let T 0 = 2. Other values for T 0 will be explored in the exercises. The number of samples N can be chosen to be whatever value one wishes as well. Because the rectangle function is symmetric about the vertical axis, the Discrete Fourier Transform formula simplifies to: F(k) = N f(τ) cos( 2π k/n τ) N τ= N 2 which also eliminates the complex variables. The sample values of the rectangle function can now be used to determine the DFT of the rectangle function for each value of k in the range N/2 to N/ For example, when N = 4, then 2 Typically, the range 0 to N 1 is used for k, but this alternative range was selected so that the DFT outputs more closely match the graph of the sinc function given earlier.

9 8 1. INTRODUCTION F(1) = = 0 cos( 2π 1/4 ( 2)) + 1/2 cos( 2π 1/4 ( 1)) 4 1 cos( 2π 1/4 (0)) + 1/2 cos( 2π 1/4 (1)) cos(π) + 1/2 cos(π/2) + 1 cos(0) + 1/2 cos(π/2) 4 = = 1 4 Graphs of the Discrete Fourier Transforms of the rectangle function with 4, 8, 16, and 32 samples are given in the right column of the figures on the next page.

10 1. INTRODUCTION τ 0 τ τ 0 τ τ 0 τ τ 0 τ 1 1 Observe that as N increases, the results of the Discrete Fourier Transform look more and more like the sinc function. But we still have not really answered the question What is the Discrete Fourier Transform?. To resolve this question, we turn to the 1807 publication by Fourier

11 10 1. INTRODUCTION that describes the construct that bears his name. It can be shown that any periodic function can be represented by an infinite series of sine and cosine functions called a Fourier Series. It turns out that as the parameter T 0 increases without bound, then nearly any function defined over a finite interval of input values can be represented by an infinite series of sine and cosine functions and the inverse of the Fourier transform f(t) = F(s) e 2πt s I ds provides this representation. It turns out that the N function samples used in the Discrete Fourier Transform can be generated by a function that is the sum of at most N sine and cosine functions called a Discrete Fourier Series. The magnitudes of these sinusoids are determined by the values found in the Discrete Fourier Transform. In the example considered in this section, the function was symmetric about the vertical axis. In this special case, the Discrete Fourier Transform is given by F(k) = N f(τ) cos( 2π k/n τ) N τ= N 2 and gives the magnitude of the sinusoid cos(k/n π) in the Discrete Fourier Series, This is illustrated in the figure on the next page. In the left column, each of the components of the Discrete Fourier Transform are shown for the case where N = 4. In the right column are the corresponding cosine functions. The function cos(k/n π) is displayed with a light line in each graph and the scaled version of the function (determined by the Discrete Fourier Transform component) is displayed with a dark line. At the bottom of the page, the complete Discrete Fourier Transform is displayed along with the result of adding all of the scaled sinusoids with the original sample values displayed on the graph. This graph represents the function f 4 (τ) = 0 cos( π τ) cos ( π 2 τ ) cos(0 τ) cos ( π 2 τ ) Observe that cos(0 τ) is equal to 1 for all values of τ and that cos ( π 2 τ) = cos ( π 2 τ) due to the symmetry properties of the cosine function. Therefore, the above function can also be represented as f 4 (τ) = cos ( π 2 τ ) which is a more compact way of expressing the Discrete Fourier Series in this case.

12 1. INTRODUCTION τ 0 τ τ 0 τ τ 0 τ τ 0 τ τ 0 τ 1 1

13 12 1. INTRODUCTION When N = 8, the Discrete Fourier Series becomes f 8 (τ) = 1 2 ( π ) cos 2 τ cos(π τ) 4 using the magnitudes of the Discrete Fourier Transform computed earlier. The function f 8 (τ) is displayed in the left graph below and the function is displayed with the original sample values in the right graph τ 0 τ 1 1 The graphs of the Discrete Fourier Series for the cases N = 16 and N = 32 are given below. Observe that as the number of samples increases, the resulting Discrete Fourier Series more closely matches the rectangle input function τ 0 τ τ 0 τ 1 1 Thus,

14 1. INTRODUCTION 13 Discrete Fourier Transform. The Discrete Fourier Transform is a method that determines the magnitudes of up to N sinusoids that can be combined and used to recover a particular sequence of N samples of some real input function over the interval T 0 /2 < x < T 0 /2. The resulting function, called a Discrete Fourier Series can be used to approximate this input function. As the number of samples N increases, the Discrete Fourier Series becomes a better and better approximation to the input signal within this interval. As the parameter T 0 increases, the Discrete Fourier Series approximates the input function over a wider interval. How much effort does it take to compute the Discrete Fourier Transform? It appears that the Discrete Fourier Transform formula must be evaluated N times and that there are N terms involved in each formula evaluation. Also, every term in the summation involves one multiplication. Thus, one is tempted to conclude that a total of N 2 multiplications and N 2 N additions are required. In 1965, a short 5 page paper written by Cooley and Tukey [11] appeared in the literature which forever changed the field of Electrical Engineering. This paper described an algorithm which computes the Discrete Fourier Transform in roughly 1/2 N log 2 (N) multiplications and N log 2 (N) additions. This algorithm is now called the Fast Fourier Transform (FFT) and significantly reduces the amount of work needed to compute the Discrete Fourier Transform for large sizes. The following figure illustrates how much the FFT improves the Discrete Fourier Transform computation. The upper line represents the number of multiplications required if the formulas prevented earlier in this section are evaluated literally. The lower line (which is somewhat difficult to distinguish from the horizontal axis) represents the number of multiplications required using the Cooley-Tukey algorithm.

15 14 1. INTRODUCTION This graph only covers FFT sizes up to 256 where the slow method requires 65,536 multiplications and the fast method requires about 1,000 multiplications. Typical FFT sizes are in the thousands or higher, but the difference in the number of operations for these sizes is so significant that one really would not be able to tell the difference between the FFT graph and the horizontal axis. One can already see the incredible improvement of the FFT in the amount of effort needed to compute the Discrete Fourier Transform. Although the Cooley-Tukey paper is one of the most influencial papers of the 20th century, it is not the first time that the technique was first described in the literature. The account of [23] shows that what is now known as the FFT may have been discovered as early as 1805 by Carl Gauss. The main difference between the success of the Cooley-Tukey paper and the unappreciated earlier papers was a new invention called a computer which could be used to actually perform the computations. Once engineers were able to compute the Discrete Fourier Transform so quickly, they found many uses for the technique. The process of creating a function which has a specified collection of function values is called interpolation. Thus, the Discrete Fourier Transform can be viewed as an interpolation process of the given sample values. The Discrete Fourier Transform can be reversed to receive the Discrete Fourier Series as an input and recover the N sample values. Because the Discrete Fourier Series is being evaluated at N points, this process is called multipoint evaluation. There are many good books (e.g. [7]) which consider the FFT from the above perspective and give algorithms that can efficiently compute the FFT. This book approaches the FFT from a different perspective which is more common among some mathematicians. The alternative perspective defines the FFT as a special case of multipoint evaluation and the inverse FFT to be an interpolation algorithm. This alternative viewpoint was first introduced in a 1971 paper by Fiduccia [15] and has since become popular with researchers who are interested in Computer Algebra and Error-Correcting Codes. This is a somewhat unfortunate development because papers are now written using both of the perspectives and it can become confusing for the beginning student to read these documents and have the same perspective on the problem as the author. A nice feature of the Fiduccia perspective is that the FFT can be developed from an algebraic perspective using the remainders that result when two polynomials are divided. This algebraic perspective has been studied in much greater detail by Daniel Bernstein whose work ([1], [2], [3]) has been very influencial of the present author s understanding of this alternative perspective of the FFT. This textbook is intended to further expand upon the ideas of these two pioneers and present the Ficuddia perspective of the FFT in a form that both undergraduate engineering and mathematics students can understand and appreciate. Because both viewpoints of the FFT are used in publications, both perspectives will be treated in this book. After some background material in Chapters 2 and 3, the FFT will be developed from the Fiduccia perspective in Chapters 4-5. Chapter 6 will then consider the problem of fast polynomial multiplication, an important application of the FFT. Next in Chapter 7, we will return to the more traditional

16 1. INTRODUCTION 15 approach to the FFT considered in this first chapter. Some additional topics will be considered in Chapters 8 and 9. EXERCISES. 1. Use a computer package to produce a graph of the rectangle function and the sinc function as shown earlier in this chapter. 2. Write a routine that computes the Discrete Fourier Transform of a function that is symmetric about the vertical axis using the formula given in this chapter. Use the routine to compute the Discrete Fourier Transform of the rectangle function. Make a graph of the Discrete Fourier Transform for N = 4, 8, 16, or 32 to verify the results given in this section. 3. Compute the Discrete Fourier Transform of the rectangle function for the case N = 128 (or even higher). Use these components to construct the Discrete Fourier Series for the case N = 128 and graph the result. The graph should really look like the rectangle function now (see below). 1 0 τ 1 However, there are spikes where the graph transitions between 0 and 1. This is called Gibb s phenonenon and is a problem that engineers must deal with when working with signals constructed from the Discrete Fourier Transform. Special components are used to prevent a signal from exceeding a certain value (called overshoot ) or going below a certain value (called undershoot ). The result of a signal going through these components might look something like the following: FIGURE 4. Throughout this chapter, assumed that T 0 = 2. Select one or more of the cases: T 0 = 1/2, T 0 = 1, T 0 = 4, T 0 = 8 and: (A) Sample the function uniformly over one period of the function using N = 32 sample values. Produce a graph of the results. (B) Compute the Discrete Fourier Transform of the sampled function and produce a graph of the results. Compare it with the graph of the sinc function produced in this chapter. In particular, how does the height of the function compare with the one produced in this chapter and how many values of the sinc function are produced between each time that the function crosses the horizontal axis?

17 16 1. INTRODUCTION (C) Try to reconstruct the rectangle function with the Discrete Fourier Series based on your Discrete Fourier Transform. Comment on the success or failure of your attempt. (D) The results of this exercise only give scaled versions of the input function. Based on you work completed for this exercise, can you guess what scaling factor (in terms of N and T 0 ) that each of these results should be multiplied by to recover the original input function?

18 CHAPTER 2 Polynomials 1. Basic operations In a typical introductory algebra course (e.g. [4]), one is introduced to the concept of a polynomial. Although polynomials involving multiple variables are frequently used in algebra, here we will restrict ourselves to polynomials involving one variable. First, let us review the concept of a monomial. Monomial. A monomial is an expression of the form a x n where a is any real number called the coefficient and n is any nonnegative integer called the exponent. Monomials can be combined to form polynomials, the topic of this chapter. Polynomial. A polynomial is either a monomial or a sum or difference of monomials. Each monomial which comprises a polynomial is called a term. Although the above definitions specify that the coefficients are real numbers, it is possible to create polynomials for other number systems. We will create polynomials with complex number coefficients in the next chapter. The degree of a monomial is simply its exponent in the case of single variables. The degree of a polynomial is the largest exponent which appears in one of its terms. The monomial which has this exponent in the polynomial is called the leading term. EXAMPLE 17

19 18 2. POLYNOMIALS A polynomial should be simplified whenever possible. This involves combining like terms or its monomials which have the same exponent. In this case, one replaces these monomials with a new term whose coefficient is the sum or difference of the coefficients of the terms with this common exponent. EXAMPLE We will now review the four basic arithmetic operations involving polynomials. The symbols f(x), g(x), h(x), etc. are used to label polynomials with variable x. Addition of polynomials. The sum of the polynomials f(x) and g(x) is determined by forming the expression f(x) + g(x) and combining like terms. EXAMPLE. Before reviewing the operation of subtraction, recall that an opposite or additive inverse of a number a is some other number b such that a + b = 0. If a is positive, then b is formed by putting a minus sign in front of a. If a is negative, then b is formed by removing the minus sign in front of a. The opposite of a polynomial f(x) is formed by replacing each term of f(x) with a term of the same degree and the opposite of the coefficient in f(x). We will denote the opposite of f(x) with the notation f(x). EXAMPLE Subtraction of polynomials. The difference of the polynomials f(x) and g(x) is determined by forming the expression f(x) + ( g(x)) and combining like terms. To multiply two monomials, we multiply the coefficients of the two monomials and add the degrees of the monomials. EXAMPLE Multiplication of polynomials. The product of the polynomials f(x) and g(x) is determined by multiplying every term of f(x) by every term of g(x) and combining like terms. EXAMPLE

20 2. THE REMAINDER THEOREM AND SYNTHETIC DIVISION 19 As one can see, polynomial multiplication requires much more effort than polynomial addition or subtraction. In Chapter 6, we will see that the FFT can be used to significantly reduce the amount of work needed to multiply two polynomials of large degree. Polynomials are an example of what is called Euclidean Domain. This means that given two polynomials a(x) and b(x), there exists a polynomial q(x) called a quotient and r(x) called a remainder such that a(x) = q(x) b(x) + r(x) with the property that either r(x) is the zero polynomial or else the degree of r(x) is less than the degree of b(x). It can be shown that q(x) and r(x) are unique in this particular type of Euclidean Domain. To determine the quotient and remainder of a(x) divided by b(x), we follow a procedure that works much like the division of two integers. Division of polynomials. The quotient q(x) of the polynomial a(x) divided by b(x) with remainder r(x) is determined by following the following sequence of steps: 1. Initialize r(x) equal to a(x). 2. While deg(r(x)) deg(b(x)): 3. Divide the leading term of r(x) by the leading term of b(x). Call this term l(x) and add it to the quotient q(x). 4. Subtract l(x) b(x) from r(x). 5. End while EXAMPLE It can be shown that there are no other polynomials q(x) and r(x) such that a = q(x) b + r(x) that have the property that the degree of r(x) is less than the degree of b(x). 2. The Remainder Theorem and Synthetic Division Suppose that we wish to evaluate a polynomial a(x) at some value ε, i.e. we wish to compute a(ε). In the previous section, we learned that there are unique polynomials q(x) and r(x) such that

21 20 2. POLYNOMIALS a(x) = q(x) (x ε) + r(x) if b(x) = x ε where either r(x) must be 0 or the degree of r(x) is less than the degree of x ε, i.e. a polynomial of degree 0. In either case, r(x) must be a constant. Let us call this constant C. If we evaluate the above equation at ε, we obtain a(ε) = q(ε) (ε ε) + r(ε) = q(ε) 0 + C = C We have proven the so-called Remainder Theorem : Remainder Theorem. If a(x), a polynomial with real coefficients, is divided by the polynomial x ε, the remainder is equal to a(ε). EXAMPLE If one divides a polynomial by another polynomial of the form x ε, i.e. a polynomial of degree 1 with a coefficient of 1 on the variable term, we do not need to write down the computations involving this variable term. This process is called synthetic division. EXAMPLE One may encounter an alternative technique for evaluating a polynomial called Horner s Rule. Horner s Rule. Given a point ε and a polynomial a(x) = a n 1 x n 1 + a n 2 x n a 2 x 2 + a 1 x + a 0, we can compute the polynomial evaluation a(ε) using the formula a(ε) = ((... ((a n 1 x) + a n 2 x) +...) + a 1 x) + a 0 EXAMPLE By comparing the above two examples, we see that evaluation of a polynomial using Horner s Rule involves the exact same computations as the Remainder Theorem implemented through synthetic division, but in a slightly different presentation.

22 3. MODULAR REDUCTION OF A POLYNOMIAL Modular reduction of a polynomial We have already learned that the Remainder Theorem can be used to evaluate a polynomial at a single point. To efficiently evaluate a polynomial at a number of points, we will use the remainders that result from other polynomial divisions. In this text, we will call the divisors in these calculations modulus polynomials and the resulting remainders will be called residue polynomials. We will also use the following notation 1 in the development of this technique. f(x) mod M(x). The modular reduction of the polynomial f(x) by the modulus polynomial M(x) is the remainder that results when f(x) is divided by M(x). In addition to the Remainder Theorem, two other results involving modular reductions will be used to efficiently evaluate a polynomial at a number of points. Modular reduction - result 1. If f(x) has degree less than M(x), then f(x) mod M(x) = f(x). This result can be established by recalling that the division of f(x) by M(x) must have a unique quotient and a unique remainder where either the remainder is zero or has degree less than the degree of M. Since f(x) has degree less than M(x), then f(x) must be this unique remainder of the polynomial division. 1 Mathematicians who have studied advanced algebra may object to this definition of f(x) mod M(x). Traditionally, the notation f(x) mod M(x) is used to represent an element of something called a quotient ring which consists of the set of all polynomials that have the same remainder as the remainder of f(x) divided by M(x). However, these mathematicians then select a representative element of this set for computational purposes. In terms of the representative elements, the definition of f(x) mod M(x) is the same as the one considered in this section. The reader should be cautioned, however, that this view is only valid with polynomials involving one variable. With polynomials involving two or more variables, one needs to learn the concept of a quotient ring and the more advanced mathematical techniques used for computing in these quotient rings. The multivariate case will not be encountered anywhere in this manuscript. More details about the relationship between residue polynomials and quotient rings is given the appendix for those with the appropriate algebra background.

23 22 2. POLYNOMIALS Modular reduction - result 2. If M A (x), M B (x), and M C (x) are polynomials such that M A = M B M C and f(x) is any polynomial, then f mod M B = (f mod M A ) mod M B f mod M C = (f mod M A ) mod M C The proof of the Result 2 is left as an exercise. These two results allow a polynomial to be evaluated using multiple modular reductions. For example, if f = X and M = (x 1) (x 2) (x 3) (x 4), then f mod M = f by the first result. By the second result, then f(1) can also be computed with the sequence of modular reductions EXAMPLE By comparing the effort needed to compute these two modular reductions with the synthetic division computed earlier, we see that we have not reduced the number of operations needed to compute the polynomial evaluation. With some additional effort, a sequence of modular reductions can efficiently evaluate a polynomial at a number of points. 4. Multipoint polynomial evaluation To evaluate a polynomial at a number of points, we can build a tree of polynomials similar to the one below for evaluating a polynomial at the points S = { 4, 3, 2, 1, 1, 2, 3, 4}. (x + 4)(x + 3)(x + 2)(x + 1)(x 1)(x 2)(x 3)(x 4) (x + 4)(x + 3)(x + 2)(x + 1) (x 1)(x 2)(x 3)(x 4) (x + 4)(x + 3) (x + 2)(x + 1) (x 1)(x 2) (x 3)(x 4) x + 4 x + 3 x + 2 x + 1 x 1 x 2 x 3 x 4 In simplified form, the polynomials are also given by

24 4. MULTIPOINT POLYNOMIAL EVALUATION 23 x 8 30x x 4 820x x x x x + 24 x 4 10x x 2 50x + 24 x 2 + 7x + 12 x 2 + 3x + 2 x 2 3x + 2 x 2 7x + 12 x + 4 x + 3 x + 2 x + 1 x 1 x 2 x 3 x 4 The multipoint evaluation algorithm works by selecting n points for which we wish to evaluate some polynomial f(x) of degree less than n. By the first result in the previous section, f mod M = f where M is the polynomial at the top of this tree. Using the second result in the previous section, one can reduce f mod M A into f mod M B where M A = M B M C at each branch of the tree. Here, M A is the polynomial at the parent node of a branch and M B and M C are the polynomials of the children nodes. For example, the first modular reduction in the above computation tree would involve the transformation of f mod X into f mod X and f mod X. Here, X and M A = M B M C. The resulting computations are COMP The modular reductions proceed down each branch of the tree until one reaches the leaf nodes at the bottom of the tree. At this point, one has computed f mod (x ε) for each of the n points ε for which we wished to evaluate f. By the Remainder Theorem, then these results represent the evaluation of f at each of the n points. The sequence of modular reductions for the case of the example can be organized into the following tree. TREE By carefully counting the number of operations involved in these modular reductions, we obtain XX additions and XX multiplications. This is the same number of operations required to compute the evaluations using synthetic division. So far, we have not gained anything by using the technique discussed in this section. Let us now rearrange the points in S according to the order S = { 4, 4, 3, 3, 2, 2, 1, 1}. The tree of polynomials now becomes

25 24 2. POLYNOMIALS (x + 4)(x 4)(x + 3)(x 3)(x + 2)(x 2)(x + 1)(x 1) (x + 4)(x 4)(x + 3)(x 3) (x + 2)(x 2)(x + 1)(x 1) (x + 4)(x 4) (x + 3)(x 3) (x + 2)(x 2) (x + 1)(x 1) x + 4 x 4 x + 3 x 3 x + 2 x 2 x + 1 x 1 or in simplified form x 8 30x x 4 820x x 4 25x x 4 5x x 2 16 x 2 9 x 2 4 x 2 1 x + 4 x 4 x + 3 x 3 x + 2 x 2 x + 1 x 1 Observe that many of the polynomials in this tree have fewer terms than the nodes of the tree given in XXX. The sequence of modular reductions for evaluating f in set S can be given by the tree PICTURE and this time only XX multiplications and XX additions are required. This is a consequence of the fact that the modulus polynomials in the new tree involve fewer terms. Note that if M B is of the form x m b and M C is of the form x m + b, then M A will be of the form x 2m b 2. Each of these polynomials has only two terms and resulted in the reduced operation count of the multipoint evaluation using the second modulus tree. We were able to arrange the points of S so that we could achieve this situation at the bottom of the modulus tree, but we were not able to construct polynomials of the desired form higher in the tree. It turns out that it is impossible to find two polynomials with two terms that multiply together to form x 2m b 2 whenever b 2 < 0 if we restrict ourselves to the real numbers. By selecting the points in S from a extension of the real numbers called the complex numbers,

26 4. MULTIPOINT POLYNOMIAL EVALUATION 25 we will be able to reduce the number of terms of the modulus polynomials higher in the tree and achieve a faster multipoint evaluation technique.

27

28 CHAPTER 3 Complex numbers 1. Number systems Over the years, mathematicians have invented several new number systems to handle cases where one cannot solve a particular problem with the existing number systems. In elementary school, one first learns the number system of the natural numbers which consists of all of the positive integers, i.e. 1, 2, 3, 4,. Next, the number zero is introduced and the number system is expanded into the whole numbers. Then, one is asked to find a number (possibly represented by ) such that + 1 = 0. Here, the concept of a negative number is introduced and the student s number system is expanded to the integers, i.e., 4, 3, 2, 1, 0, 1, 2, 3, 4,. Later, one is asked to find a solution to an equation similar to 2x = 1 and the concept of a fraction is introduced. Now, the student s number system has been expanded to the rational numbers, i.e. all numbers that can be expressed as a ratio of two integers. An equation similar to x 2 2 = 0 cannot be solved with the rational numbers and the concept of an irrational number is introduced. The number system is next expanded to the real numbers. The number system must be expanded again so that an equation of the form x 2 +1 = 0 can be solved. We will introduce a new symbol, 1, that we will define as the solution to this equation. Mathematicians traditionally use the symbol i to represent 1 which engineers typically choose the symbol j to represent the symbol. In this book, we will use the symbol I which is used in some popular computer algebra packages. The number system has now been expanded to the complex numbers 27

29 28 3. COMPLEX NUMBERS Complex numbers. The number system of complex numbers consists of all expressions of the form A + I B where A and B are real numbers. We can extract the two components of a complex number using the following operations. Components of a complex number. If C = A + I B is a complex number, then Re(C) = A is the real part of the complex number and Im(C) = B is the imaginary part of the complex number. EXAMPLE It is unfortunate that one of the components of a complex number is called imaginary. This term was introduced because at first some people did not believe that these numbers had any practical applications. If one thinks about it carefully, negative numbers can also be considered imaginary because they cannot be used to count anything tangible in the real world. However, negative numbers have become accepted because they can be used to represent the concepts of debt and loss. This text is all about one of the important practical application of complex numbers. So, while the term imaginary is traditionally applied to one of the components of a complex number, this term should not be interpreted as a description of the usefulness of complex numbers. So what is the next expansion after the complex numbers? A consequence of the Fundamental Theorem of Algebra introduced by Carl Gauss states that the complex number system is complete, meaning that all equations with coefficients in the complex numbers can be solved using only the complex numbers. It is possible to expand the complex number system two more times, but each time we lose a property that one usually associates with numbers. First, the quaternions are a set of numbers of the form A+I B + J C + K D where A, B, C, and D are real numbers and I 2 = J 2 = K 2 = 1 and I J K also equals 1. This number system is not commutative, which means that a + b can be different from b + a. This number system was long thought to be of only theoretical interest, but has recently been applied to computer graphics and video games. This number system can again be expanded into the octonions which are like the quaternions, but has eight components. This number system is also not commutative, but has the additional property that it is not associative. That is to say, a + (b + c) may be different from (a + b) + c. Currently, this number system seems to be mainly of

30 2. COMPLEX ARITHMETIC 29 theoretical interest. It turns out that the octonions represent the last expansion of the number system that can be made where both addition and multiplication are defined. The following table summarizes the various expansions of the number system discussed in this section. TABLE 2. Complex arithmetic In the previous section, we learned that a complex number is of the form A+I B where A and B are real numbers. The two quantities A and I B are kept distinct because one cannot combine real and imaginary numbers. In [30], Loy relates the components of a complex number to the idea of apples and oranges in a fruit basket. Apples can be added to apples and oranges can be added to oranges, but apples cannot be added to oranges. This illustration may be useful as we give the following definition of addition and subtraction with complex numbers Complex addition. The sum of the two complex numbers A+I B and C + I D is given by (A + C) + I (B + D). EXAMPLE The sum of 1 + I 2 and 3 + I 4 is given by: (1 + 3) + I (2 + 4) = 4 + I 6 Complex subtraction. The difference of the two complex numbers A + B I and C + D I is given by (A C) + (B D) I. EXAMPLE Complex multiplication works by treating each complex number as a binomial (a polynomial with two terms) and combining the four terms of the product using the property that I 2 = 1. In other words: Complex multiplication. The product of the two complex numbers A + B I and C + D I is given by (AC BD) + (AD + BC) I EXAMPLE

31 30 3. COMPLEX NUMBERS Before discussing complex division, we first introduce the concept of the complex conjugate Complex conjugate. The conjugate of the complex number A + B I is given by A B I Complex division. The quotient of the two complex numbers A + B I and C + D I is given by A + B I C + D I AC + BD BC AD = C 2 + I D2 C 2 D 2 This division formula is established by multiplying the numerator and denominator of the quotient by the complex conjugate of C + D I and simplifying. The complex numbers can be graphed on a Cartesian plane where the horizontal axis is used for the real component and the vertical axis is used for the imaginary component. The complex number A + B I is mapped on the complex plane using the ordered pair (A, B). Im (A, B) B A Re EXAMPLE. 3. Polar Representation of Complex Numbers In this section, we will consider a second method of representing a complex number called polar form illustrated in the figure below.

32 3. POLAR REPRESENTATION OF COMPLEX NUMBERS 31 Im r (r, θ) θ Re Complex numbers (polar form). The number system of complex numbers consists of all expressions of the form (r, θ) where r is the distance from the point to the origin called the magnitude and θ is the angle that a line drawn from the point to the origin makes with the real axis. This angle is traditionally called the angle or argument of the complex number. If r = 0, then θ is undefined as any angle can be used to represent this point. One difficulty with working with complex numbers in polar form is that they may have more than one representation. If the above complex number is (r, 40 o ), then a second representation of the number is (r, 40 o +360 o ) = (r, 400 o ). In fact, any multiple of 360 degrees can be added or subtracted from the argument to obtain another polar representation of the complex number. To make sure that every complex number has a unique representation in polar form, one often restricts the range of argument values allowed. In this text, we will assume that the angles are in the range 0 o θ < 360 o or 0 θ < 2π in radians. If a computation results in a value of θ outside of this range, multiples of 360 o will be added or subtracted from the result to bring θ back within 0 θ < 360 o. We are now going to develop methods of converting between the Cartesian representation and the polar representation of a complex number. The following figure which illustrates both representations of a complex number may be helpful in developing these formulas.

33 32 3. COMPLEX NUMBERS Im θ r A B Re First, we will show how to convert from the Cartesian representation to the polar representation. Observe that r can be computed using the Pythagorean Theorem or the distance formula r = A 2 + B 2 and θ satisfies θ = tan ( ) B A To solve this equation for θ, one must exercise caution because tan 1 is restricted to the range 180 o < θ < 180 o. The following formulas give a solution for θ in the range 0 θ < 360 o for all cases except when r = 0. θ = tan 1 (B/A) if A > 0 and B > 0 tan 1 (B/A) o if A < 0 tan 1 (B/A) o if A > 0 and B > 0 0 if A > 0 and B = 0 90 o if A = 0 and B > o if A < 0 and B = o if A = 0 and B < 0 undefined if A = 0 and B = 0 To convert from polar form to Cartesian form, we will apply the following results from trigonometry.

34 3. POLAR REPRESENTATION OF COMPLEX NUMBERS 33 A = r cos(θ) B = r sin(θ) Substituting these formulas into A + B I, we obtain another form of the polar representation of a complex number r (cos(θ) + I sin(θ)) which is sometimes abbreviated as r cis θ. Engineers often instead use the abbreviation r θ and call the polar representation of a complex number a phasor. One advantage of the polar representation of complex numbers is that multiplication is easy in this form. Complex multiplication (polar form). The product of two complex numbers written in polar form r 1 θ 1 and r 2 θ 2 is given by (r 1 θ 1 ) (r 2 θ 2 ) = r 1 r 2 (θ 1 + θ 2 ) Thus, all one needs to do is multiply the magnitudes and add the arguments of two complex numbers to compute the product. This result is a consequence of the sum and difference formulas learned in trigonometry. The derivation of the formula is left as an exercise. This multiplication formula can be used to derive an expression for the square of a complex number (r θ) 2 = r 2 (2θ) By repeated use of the multiplication formula, we can derive de Moivre s Theorem which allows a complex number to be raised to any integer power n. de Moivre s Theorem. If r θ is a complex number with magnitude r and argument θ, then (r θ) n = r n (nθ) By a similar derivation used to produce the formula for complex multiplication, a formula for the division of r 1 θ 1 and r 2 θ 2 is given by

35 34 3. COMPLEX NUMBERS Complex division (polar form). The product of two complex numbers written in polar form r 1 θ 1 and r 2 θ 2 is given by r 1 θ 1 r 2 θ 2 = r 1 r 2 (θ 1 θ 2 ) There are no good formulas for addition and subtraction in polar form. The best course of action for these operations is to convert the two numbers to be added or subtracted into Cartesian form, compute the sum or difference, and then convert the result back into polar form if desired. 4. Primitive Roots of Unity In this section, we will restrict ourselves to complex numbers which have magnitude 1 when represented in polar form. These complex numbers are said to form the unit circle in the complex plane. Im(z) 1 Re(z) Consider the equation z n = 1. Here, z is a complex variable, i.e. an expression of the form x + I y where x and y are unknown real numbers. Alternatively, z can be a variable involving complex numbers represented in polar form. A solution to the above equation is called an nth root of unity. By de Moivre s Theorem discussed in the previous section, we can verify that 1 (360 o d/n) = cos(360 o d/n) + I sin(360 o d/n) is a solution to this equation for all 0 d < n. The Fundamental Theorem of Algebra tells us that there are exactly n solutions to the equation and we see that the above expression can be used to find all of them.

36 4. PRIMITIVE ROOTS OF UNITY 35 EXAMPLE For n = 8, the solutions to z 8 = 1 are: 1 (360 o 0/8) = 1 0 o 1 (360 o 1/8) = 1 45 o 1 (360 o 2/8) = 1 90 o 1 (360 o 3/8) = o 1 (360 o 4/8) = o 1 (360 o 5/8) = o 1 (360 o 6/8) = o 1 (360 o 7/8) = o 1 90 o o 1 45 o o 1 0 o o o o An element ω is said to be a primitive nth root of unity if ω n = 1, but ω m 1 for any m that divides n. Another way of looking at a primitive root of unity is that {ω 0, ω 1, ω 2,...,ω n 1 } should give all of the solutions to x n = 1. EXAMPLE If n = 8, then a primitive 8th root of unity is given by ω = 1 45 o. In the above example, we saw that {ω 0, ω 1, ω 2,..., ω 7 } gives all eight solutions to z 8 = 1. On the other hand, the element ω 1 = 1 90 o is not a primitive root of unity. Note that (ω 1 ) 4 = 1 where 4 divides 8. Also, {ω 0 1, ω 1 1, ω 2 1,...,ω 7 1 } only gives four of the solutions to z 8 = 1: {1 0 o, 1 90 o, o, o }. It can be shown that if ω is a primitive nth root of unity, then ω 1 = ω n 1 is also a primitive nth root of unity. Here, the notation ω 1 means the (multiplicative) inverse of ω and is the unique element such that ω ω 1 = ω 1 ω = 1.

37 36 3. COMPLEX NUMBERS EXAMPLE Observe that (1 45 o ) (1 315 o ) = (1 360 o ) = 1. So, o is the multiplicative inverse of 1 45 o. So, o is also a primitive 8th root of unity. One can verify this by moving counterclockwise in increments of 45 degrees in the figure above. Next, consider the equation z n = 1 θ. A solution to the above equation is called an nth root of 1 θ. Again, by de Moivre s Theorem, we can verify that 1 (θ/n o d/n) = cos(θ/n o d/n) + I sin(θ/n o d/n) is a solution to this equation for all 0 d < n. This gives all n solutions to this equation. EXAMPLE Consider the equation z 4 = 1 = o. The four solutions to the equation are given by 1 (180 o / o 0/4) = 1 45 o 1 (180 o / o 1/4) = o 1 (180 o / o 2/4) = o 1 (180 o / o 3/4) = o o 1 45 o o o

38 5. EULER S FORMULA 37 Finally, the solutions to z 4 1 are the called the 4th roots of unity and are given by {1, I, 1, I}. I I Here ω = I is a primitive 4th root of unity. Since I 4 = 1, then the powers of I cycle according to the following pattern: ω 0, ω 4, ω 8,... = 1 ω 1, ω 5, ω 9,... = I ω 2, ω 6, ω 10,... = 1 ω 3, ω 7, ω 11,... = I 5. Euler s Formula In a typical Calculus course, one encounters the following Taylor expansions for the cosine, sine, and exponential functions: cos(x) = 1 x2 2! + x4 4! x6 6! + sin(x) = x x3 3! + x5 5! x7 7! + e x = 1 + x 1! + x2 2! + x3 3! + x4 4! + x5 5! + x6 6! + x7 7! + In the 18th century, Leonhard Euler saught a single formula that related these three expressions. The expression cos(x) + sin(x) = 1 + x 1! x2 2! x3 3! + x4 4! + x5 5! x6 6! x7 7! + almost does it, but some of the signs are wrong.

39 38 3. COMPLEX NUMBERS In order to make a formula that works, Euler decided to replace the x in the definition of e x with I y where y is a real number. Then we obtain e I y = 1 + (I y) 1! = 1 + I y 1! = 1 + I y + (I y)2 2! + I2 y 2 2! + (I y)3 3! + I3 y 3 3! + + I4 y 4 4! (I y)4 4! y2 1! 2! I y3 + y4 3! 4! + I y5 ) ( 5! = (1 y2 2! + y4 I y 4! + + I y3 3! ) = (1 y2 2! + y4 4! + + I = cos(y) + I sin(y) + + I5 y 5 5! I y6 6! + I y5 5! (I y)5 5! 1! ( y 1! y3 3! + y5 5! I6 y 6 6! + ) + ) (I y)6 6! + + Now, certain mathematicians would correctly raise several objections to the derivation of the above formula. Euler did not concern himself with such issues and neither will we in this presentation. However, it should be mentioned that some more advanced mathematics covered in a course in complex variables is needed to properly derive the above result. In any event, Euler s Formula. e I y = cos(y) + I sin(y) is now widely accepted by mathematicians and engineers. Another way of representing a complex number in polar form which uses Euler s Formula is given by r (cos(θ) + I sin(θ)) = r e I θ and the nth roots of unity are sometimes expressed as {e I 2π/n, e I 4π/n, e I 6π/n,..., e I 2(n 1)π/n } in this form. Here, we used the fact that 360 degrees is equivalent to 2π radians. Two primitive nth roots of unity are given by e I 2π/n and e I 2π/n in this form. 6. Rotation transformations Suppose that we multiply a complex number r θ by I = 1 90 o. What is the effect of this multiplication? From the multiplication formula for complex numbers