A method for the easy storage of discriminant polynomials by RANAN B. BANERJI Case Western Reserve University Cleveland, Ohio I~TRODUCTIO~ One olthepurposes of. feature extraction is to save_ on the memory required to store the descriptions of the patterns learned. It is, however, the opinion of this author that a far more important function of feature extraction is to attach statistical significance to the patterns learned l and to change the measured variables for future experiments in the same pattern recognition environment. The present paper provides a method for fulfilling the above first or "simplification" aspect of feature extraction. Unfortunately, the nature of the method is such that very little light seems to be cast on the latter "significance" aspect of feature extraction. Since the method involved deals heavily on the theory of finite fields, an area of mathematics not of wide usage in pattern recognition, we shall include in the next section a short tutorial on the subject together ",ith an explanation of the method. The third section will discuss some of the algorithms involved and give an estimate on the memory saving and the computational work involved. FIELDS A field is a set of elements on which addition, subtraction and multiplication can be carried out and division by any non-zero element is possible. Fractions and real numbers are examples of fields. Positive integers are not fields since 4 cannot be subtracted from 2. Positive and negative integers do not form fields either since 3 cannot be divided by 2. Fractions and reals are infinite in number. On the other hand, a good example of a finite field is the set of integers {O, 1, 2,... (p-l)} (where p is some prime number) in which all addition and multiplication is carried out "mod p." That is, the sums and products, if they exceed (p-l), are divided by p and the remainder taken as the result. As an example, the field of integers mod 3 have the following "addition" and "multiplication" tables +10 1 2 ~ 'i I 'i ; ~ 2 120 1. 10 1 2 000 0 101 2 202 1 Finite fields having q elements exist only if q is prime or is an integral power pn of some prime integer p. These latter fields, however, do not have the "modulo q" structure that the prime fieids have-. Fo~ fnstance, division by 2is-i~possibie- modulo 8-no integer less than 8, multiplied by 2, yields 3, for instance, sho",ing that ~2 is undefined modulo 8. The construction of the addition and multiplication tables of such "prime-pmver" fields need some detailed explanation, which we proceed to give below. The methods are intimately tied to polynomials, the major topic of this paper. Given any field F, one can form polynomials in a "variable" x with coefficients from F. Polynomials are added and subtracted "componentwise" as usual. That is, the sum of and IS ao+alx+a2x2... anxn 7 Note that both an and b n need not be non-zero-we have made the "degrees" equal only for convenience. The next thing to be noted is that for our present purposes, a polynomial can be merely considered as n-tuples of field elements, the x being a mere "placeholder." However, the polynomial format for exhibiting the n-tuples takes great mnemonic significance,,,hen one defines multiplication of polynomials by saying that the product of the two polynomials above is aobo+ (a1bo+aob1)x+ (~bo+albl+aob2)x2+... anbnx 2n yielding a 2n-tuple. Polynomials do not form fields: although addition and subtraction is possible, division cannot always be performed: 2+x, for instance, cannot be divided by 3+x. However, just as finite fields can be produced by performing all operations modulo a prime, polynomial fields can also be formed by performing addition and (especially) multiplication module a prime or irreducible polynomial, i.e., one which cannot be factored into polynomials with coefficients from the same field. In the field modulo 3, for instance we find that (x+l) (x+l) =x2+2x+l (x+l) (x+2) =x 2 +2 (x+2) (x+2) =x2+x+l 497
498 National Computer Conference, 1973 and these are the only polynomials which can be factored into other polynomials (we are neglecting the cases where the coefficients of the highest power of x are not 1, since one can clearly "divide these out," i.e., 2x+1 is the same as 2(x+2)). Hence, x2+x+2 (or more conveniently written x 2-2x-1 for future purposes; note that since 1+2=0, 2= -1) is an irreducible polynomial. We give below some examples of operations on polynomials mode X2 = 2x+ 1. These operations will be restricted to polynomials whose degree is less than 2 and whose coefficients, clearly, come from the field of integers mod 3. As can be seen, there are nine (3 2 ) such polynomials, all the way from O=O+Ox to 2+2x. (l+x) (2+x) =2+X2 which, on division by x 2-2x -1 yields 2 + 1 + 2x = 2x. The major fact which we shall use for our purposes here is that in any finite field all the non-zero elements can be obtained by raising some element of the field to successively higher powers. For instance, the polynomial 2+2x, raised to successive powers, yields all the 3 2 elements of the field mod x 2-2x-1 and 3. To illustrate, (2+2x)2=4+2x+x 2 = 1+2x+ (1+2x) =2+x (2+2x)3= (2+x) (2+2x) = 1+2x2 =1+2(1+2x) =x and similarly mod x2-2x-1, 3 mod x2-2x-1, 3 (2+2x)4=x(2+2x) =2x+2(1+2x) =2 mod x2-2x-1, 3 (2+2x)5 =l+x mod x2-2x-1, 3 (2+2x)6 =1+2x (2+2x)7 =2x (2+2x)8 =1 Two things are to be noted about this table which shows the eight polynomials as powers of x. The first is that the elements 1 and 2 behave just like they did in the field mod 3, i.e., 2X2 = (2+2x)4(2+2x)4 = (2+2x)8 =1 These form a sub field-and they occur in the right places (2= (2+2x)4) to make this possible. This is true in any field having qn elements, as we shall illustrate further as we go on. The second important thing to be noted is that the polynomial2+2x actually does "satisfy" the equation x2-2x-1, i.e., if its value is "plugged in" for x in the above polynomial, the result is (2 + 2x) 2-2 (2 + 2x) -1 = 2 + 2x+ (2 + 2x) + 2 = O! As a matter of fact, x also satisfies x 2-2x-1 as can be seen from the above table; also the elements of the field can be obtained by successive powers of x itself since X= (2+2x)3; x 2 = (2+2x)6=1+2x, x 3 = (2+2x)9=2+2x and so on. Not all irreducible polynomials have the property that the successive po wers of any of its solutions generate all the nonzero elements of a field. The polynomial X2+ 1 is irreducible in the field mod 3 and yet the only two elements which satisfy it are 2 and 1 whose successive powers merely generate a subfield. N ow if we can find an irreducible polynomial of degree n in a field of q elements, such that its solution generates all the qn polynomials of degree less than n by its successive powers (such an element is called a primitive element), then we can express any polynomial by an integer-its "logarithm" with respect to the primitive element (which can safely be taken to be x). In the above field 2+2x could be represented by 3 and its value as a polynomial could be obtained by dividing x 3 by X2-2x -1 yielding, as expected, 2 + 2x as a remainder 2x 2-4x-2 2x+2 (x+2 The memory saved by using this technique will be discussed in the next section. Our "trick" for storage of a large number of polynomials involves the discovery of an irreducible polynomial and an element of the field mod this polynomial which is primitive, i.e., whose successive powers generate all the polynomials in the field. For instance, in the field of polynomials ",ith coefficients in the mod 3 field, taken modulo X2+ 1, we see that x+1 is primitive, since (x+1)2=x 2 +2x+ 1 = 2x; so (x+1)2-2(x+1)-1=2x-2-1=0, i.e., that x+1 "satisfies" X2+ 2x+1, and therefore behaves just like 2+2x in the field of polynomials mod x 2-2x -1. Once such a primitive element Cl is found, any polynomial can be represented by its "logarithm" "ith respect to Cl. In the field mod x2+1, for instance, 2x+2= (X+1)5 as can be seen by carrying out the follo\\ing calculation (x+1)2=2x :. (x+1)4= (2X)2=X2= -1=2 :. (x+1)5=2(x+1) =2x+2 It will be noted how the fourth power of the primitive element is again 2, just as in the case modulo x 2-2x-1. In the next section we shall discuss an algorithm for finding irreducible polynomials and discuss possible ways of finding primitive elements of polynomial fields and the manipulations needed for converting an integer to its "exponential." In the process we shall be exemplifying a process of constructing fields of polynomials in more than one variable. COMPUTATIONAL METHODS Knuth and Alanen 4 have given a method for finding irreducible polynomials with coefficients in a prime field such that its solution is primitive in the polynomial field. The method generalizes to prime-power fields also. Basically it is a search method-but there are two restrictions which reduce the search somewhat.
A Method for the Easy Storage of Discriminant Polynomials 499 The first restriction states that the constant term in any polynomial of this nature must be either a primitive element of the coefficient field (if the degree is even) or the negative of one (if the degree is odd). After a polynomial of this nature is chosen, one can raise the variable to successive po wers modulo the polynomial (qn-l) /q-l times (notice that (3 2-1/(3-1) =4 and xi=2 mod x2-2x-l). If the last operation yields a primitive element of the coefficient field, then the structure of the polynomial field is known. This latter test is somewhat laborious; however, the test can be shortened somewhat if the polynomial is tested for irreducibility first. In this case, one can try any polynomial (including the variable) for being primitive by raising it to successive powers to the largest divisor of (qn -1) / (q -1). It may be possible, given any prime polynomial u(x), to calculate directly a primitive polynomial in the field modulo u: but this author is not aware of any such method. Testing a polynomial for irreducibility in a finite field can be carried out by using a modification of the Berlekamp2 method suggested by Knuth. 5 The method is described by him for a prime field, but can be applied to prime power fields also. We discuss this in what follows, using the nineelement polynomial field of the previous section as the coefficient field. Since the elements of this field are polynomials in x, the results of this section will illustrate the use of the method to polynomials in more than one variable. Since there are 8 linear polynomials of the form y+a(x) when a(x) is some element of the field of 9 elements, they are (8X4)/2= 16 factorable quadratic polynomials whose constant terms are primitive elements of the coefficient field (only a, a 3, a 5 and a 7 are primitive). There are a total of 9 X 4 = 36 quadratics with coefficient for y2 and a primitive constance term. Hence, the probability that a quadratic "\vith a primitive constant chosen at random will be factorable is 1%2 ~~. Hence the chances are less than 3 percent that an irreducible polynomial will not be found in 5 trials. Also, since 80 (9 2-1) is divisible by 2, 4, 5, 8, 10, 16, 20 and 40, 72 of the two-variable polynomials in the resulting field,vill be primitive-so the chance of one chosen at random being primitive is large indeed. Once a primitive element can be found, one can find another polynomial 'v-hich it satisfies by testing its successive powers for linear independence (the way we found x 2-2x-l for x+ 1 in Section 2 in the field modulo x+ 1), and this new equation (which, according to the theory of fields is bound to be irreducible) will have y itself as a primitive element. Let us now describe the test for irreducibility for a polynomial u (y). \Ve shall assume for simplicity that u (y), if it is factorable at all, has no repeated factors. This can be tested quite easily by seeing if u(y) and (du)/(dy) have any common factor by taking their greatest common divisor by the usual method. If this g.c.d. is not 1, then it has repeated factors and hence is not irreducible. If it does have a g.c.d. of 1, we use the follo\ving factorization technique. Take the polynomials J;(y) =yi_y modulo u(x) for i= 1, 2, '" [n/2] where [n/2] is the largest integer less than [n/2]' u (y) is irreducible if the g.c.d. of Ii (y) and u (y) is 1 for every i, It is not as hard as it seems to calculate the polynomials Ii(Y) as the large values of qi might indicate. We can see this as follows. Suppose there was a matrix Then if there is a polynomial Q = qooqoi... qo,n-1 qn-i,o.., qn-l,n-l mod u(y) w(y) =ao+aly+... an-iyn-1 Then w(y) q can be readily seen to be since all the product terms in the expansion are multiples of q and q=o mod q and u(y). Also, since each a is a power at of some primitive element a q = (a q ) t= 1 so that replacing x qi by n-l L qijx i i=o it can be seen that w(y) q can be obtained by multiplying the transpose of vector (an-i, a n-2... ai, ao) by Q. Calculating the matrix Q is not such a difficult thing either. If u(y) =UO+UlY+... un_iyn-l+yn and w(y) is a polynomial of degree less than n then the result of multiplying w(y) by y and dividing by u(y) leaves a remainder -UOWn-l+ (WO-UIWn-I)Y+... (Wn_2-Un_lWn_l)yn-l which can be obtained by multiplying the transpose of the vector (Wn-l,... WI, WO) by the matrix T= -un-ii 0 O... 0 -Un-2 0 1 O... 0 -Ul 0 0 O.,,1 -Uo 0 0 O..,0 Hence, to find the successive polynomials x q, x 2q we merely take the transpose vectors (0, 0,..., 1) and multiply successively by T. Raising T to the qth power is accomplished most readily by expanding q in binary. If q=9, then P= P. T and T8= T4T4 when T4= T2T2 so that a total of 4 multiplications suffice. Let us now exemplify the method by testing y2_y_x= u(y) for irreducibility. vve are taking x to be the primitive element of the field of the polynomials of degree less than 2 and coefficients from the field mod 3. Its addition and multi-
500 National Computer Conference, 1973 plication table can be gleaned from the previous section. We first notice that the derivative of u(y) to be 2y-l (this derivative also must be taken mod the field of 9 elements also-in this case it makes no difference). 2y-l and y2_ y-x have no common factor as can be seen by the following g.c.d. calculation in the field of 9 polynomials 2y+2 2y-l )y2_y - x y2-2y y-x y-2 2-x xy 2-x ) 2y-l 2y Notice that 2-x=2+2x=x 3 and 2=x 4 in the 9-element field so that (2-x) x=2. We now set up the matrix 1 1 T= x 0 and raise it to the power 9 and finally l+x 1 x 7 1 T2= - X X X x x6+x rx 2 T4= X 6 +X 2 x 3 1 x6+x 5 T8= X 6 +X 3 x5+x 2 _1x4x7 x 5 +1-11 x2 x4+1 x 0 x4 T9= l+x 3 1 x 5 1 Thus the successive rows of the matrix Q are obtained by multiplying the transpose of (0, 1) giving the matrix, written left to right instead of bottom to top. This yields o 1 Q= 1 x4 y9 therefore is y+x4 mod y2_y_x. The g.c.d. of y+2 and y2_y-x is 1, hence y2_y-x is irreducible. To test whether y is a primitive element of the field modulo y2_y-x we note that (9 2-1)/(9-1)=10, if the first five powers of yare not primitive polynomials in x (in the field mod x2-2x-l) then y is primitive. The successive po"wers of y can be found by multiplying the transpose of (0, 1) by the matrix -1 as before, 3 yielding the five polynomials y, x+y, (l+y)y+x, (1+2x)y+l, (2+2x)y+ (2+2x) so that y is indeed primitive. Also, y5= (2+2x)y+ (2+2x) = x 3 Y+X 3 and so yio=x6y2+x6+2x6y=x6(y+x) +x6+x 2 y= x 7 x6+ (x 2 +x6)y=x5+ (1+2x+2+x) =x 5, a primitive element of the coefficient field. What would be the logarithm of y+x+2=y+x6? Since X+y=y2 we can write y+x 6 =X 5 (X+Y) + (1-x5)y=x5y2+x7y =y(x 7 +x5y) =y(x6(x+y) + (X 5 _X 6 )y) =y2(x 7 +X 6 y) = y2(x 6 (X+Y)) =y4x 6 N ow since x 5 = yio, then x 6 = (x 5 ) 6 since x 30 = x8.3+6 = 1 3.x 6 Hence, x6=y60 and therefore y+x+2=y64 and the integer 64 is stored to represent the polynomial. At retrieval time we recall that y64=y60.y4 and y60=x 6. y4 can be obtained by the matrix multiplication to yield x2y+ 1 as above (or by straight division of y4) so that y64 is found to be equivalent to x 6 (x 2 y+l) =y+x 6 =y+x+2. It ought to be borne in mind, of course, that these operations all have to be recursive, 'with one level of recursion for each variable involved. The above calculations give the reader some idea of the amount of calculation involved. The initial discovery of the irreducible polynomials and the primitive elements is a "once-for-all calculation" and hence not of great importance. However, the work consists of setting up the matrix T and then the n successive multiplications of vectors to obtain the Q matrix. Getting Tq only takes at most 2 10~(n+1) multiplications. The test of the primitiveness of the variable need not be done by successive (qn-l) / (q-l) multiplicationsonly the factors of this number are important. Thus in our example we needed only calculate yt5 for our test. The "exponentiation" method is equally efficient. However, at present we have no good estimate of the effort involved in the finding of the "logarithm." It must be recalled that after the discriminant polynomials are learned, these are "once-forall" operations also. The memory saving, however, is considerable. Suppose we want to store N polynomials in V variables and the highest degree to which any variable is involved is n. Let p be the smallest prime larger than all coefficients, considered integral. Then to apply our method we need V irreducible polynomials, each of degree n, needing the storage of n V coefficients (recall that any polynomial which is used as a coefficient can be stored as its logarithm and hence its own coefficients need not be stored). We might perhaps need the storage of some of the (qn -1) / (q -1) powers of some of the primitive elements as elements of the coefficient fields-but these can also be sorted as the integer "logarithms." Thus a maximum of (n+ 1) V + N integers need be stored. This is much less than N (n+ 1) v coefficients to be stored initially. ACKNOWLEDG:\IENTS The research that led to the preparation of this paper was supported by the Air Force Office of Scientific Research under Contract Xumber AFOSR 71-2110B.
A Method for the Easy Storage of Discriminant Polynomials 501 REFERE~CES 1. Banerji, R. B., "Some Linguistic and Statistical Problems in Pattern Recognition," Pattern Recognition, 3,409,1971. 2. Berlekamp, E. R., "Factoring Polynomials over Finite Fields," Bell System Technical Journal, 46, 1853, 1967. 3. Elspas, B., "The Theory of Autonomous Linear Sequential Networks," Trans. IEEE, PGCT-6, 45,1959. 4. Knuth, E., Alanen, J. D., Tables of Finite Fields, Sankhya, Ser A, 26,305,1964. 5. Knuth, D. E., Seminumerical Algorithms, Addison-Wesley, New York, p. 381, 1971.