Why Bernstein Polynomials Are Better: Fuzzy-Inspired Justification

Transcription

1 Why Bernstein Polynomials Are Better: Fuzzy-Inspire Justification Jaime Nava 1, Olga Kosheleva 2, an Vlaik Kreinovich 3 1,3 Department of Computer Science 2 Department of Teacher Eucation University of Texas at El Paso 500 W University El Paso, TX 79968, USA 1 navajaime@gmailcom 2,3 {olgak,vlaik}@utepeu Abstract It is well known that an arbitrary continuous function on a boune set eg, on an interval [a, b] can be, with any given accuracy, approximate by a polynomial or by a piece-wise polynomial function spline) Usually, polynomials are escribe as linear combinations of monomials It turns out that in many computational problems, it is more efficient to represent each polynomial as a Bernstein polynomial eg, for functions of one variable, a linear combination of terms x a) k b x) n k In this paper, we provie a simple fuzzybase explanation of why Bernstein polynomials are often more efficient than linear combinations of monomials, an we show how this informal explanation can be transforme into a precise mathematical explanation I INTRODUCTION: POLYNOMIAL AND BERNSTEIN APPROXIMATIONS Functional epenencies are ubiquitous Several ifferent quantities are use to escribe the state of the worl or a state of a system in which are are intereste For example, to escribe the weather, we can escribe the temperature, the win spee, the humiity, etc Even in simple cases, to escribe the state of a simple mechanical boy at a given moment of time, we can escribe its coorinates, its velocity, its kinetic an potential energy, etc Some of these quantities can be irectly measure an sometimes, irect measurement is the only way that we can etermine their values However, once we have measure the values of a few basic quantities x 1,, x n, we can usually compute the values of all other quantities y by using the known epenence y = fx 1,, x n ) between these quantities Such functional epenencies are ubiquitous, they are extremely important in our analysis of real-worl ata Nee for polynomial approximations With the large amount of ata that are constantly generate by ifferent measuring evices, most of the ata processing is performe by computers So, we nee to represent each known functional epenence y = fx 1,, x n ) in a computer In a computer, among operations with real numbers, the only ones which are irectly harware supporte an are therefore extremely fast) are aition, subtraction, an multiplication All other operations with real numbers, incluing ivision, are implemente as a sequence of aitions, subtractions, an multiplications Also harware supporte are logical operations; these operations make it possible to use several ifferent computational expressions in ifferent parts of the function omain For example, computers use ifferent approximation for trigonometric functions like sinx) for x [ π, π] an for inputs from the other cycles Therefore, if we want to compute a function fx 1,, x n ), we must represent it, on each part of the omain, as a sequence of aitions, subtractions, an multiplications A function which is obtaine from variables x 1,, x n an constants by using aition, subtraction, an multiplication is nothing else but a polynomial Inee, one can easily check that every polynomial can be compute by a sequence of aitions, subtractions, an multiplications Vice versa, by inuction, one can easily prove that every sequence of aitions, subtractions, an multiplications leas to a polynomial; inee: inuction base is straightforwar: each variables x i is a polynomial, an each constant is a polynomial; inuction step is also straightforwar: the sum of two polynomials is a polynomial; the ifference between two polynomials is a polynomial; an the prouct of two polynomials is a polynomial Thus, we approximate functions by polynomials or by piecewise polynomial functions splines) Possibility of a polynomial approximation The possibility to approximate functions by polynomials was first proven by Weierstrass long before computers were invente) Specifically, Weierstrass showe that for every continuous function fx 1,, x n ), for every box multi-interval) [a 1, b 1 ] [a n, b n ], an for every real number ε > 0, there exists a polynomial P x 1,, x n ) which is, on this box, ε-close to the original

2 function fx 1,, x n ), ie, for which P x 1,, x n ) fx 1,, x n ) ε for all x 1 [a 1, b 1 ],, x n [a n, b n ] Polynomial an piece-wise polynomial) approximations to a functional epenence have been use in science for many centuries, they are one of the main tools in physics an other isciplines Such approximations are often base on the fact that most funamental physical epenencies are analytical, ie, can be expane in convergent Taylor polynomial) series Thus, to get a escription with a given accuracy on a given part of the omain, it is sufficient to keep only a few first terms in the Taylor expansion ie, in effect, to approximate the original function by a piece-wise polynomial expression; see, eg [1] How to represent polynomials in a computer: traitional approach A schoolbook efinition of a polynomial of one variable is that it is a function of the type fx) = c 0 + c 1 x + c 2 x c x From the viewpoint of this efinition, it is natural to represent a polynomial of one variable as a corresponing sequence of coefficients c 0, c 1, c 2, c This is exactly how polynomials of one variable are usually represente Similarly, a polynomial of several variables x 1,, x n is usually efine as linear combination of monomials, ie, expressions of the type x 1 1 x n n Thus, a natural way to represent a polynomial fx 1,, x n ) = c 1 n x 1 1 xn n 1,, n is to represent it as a corresponing multi-d array of coefficients c 1 n Bernstein polynomials: a escription It has been shown that in many computational problems, it is more efficient to use an alternative representation This alternative representation was first propose by a mathematician Bernstein, an so polynomials represente in this form are known as Bernstein polynomials For functions of one variable, Bernstein propose to represent a function as a linear combination c k x a) k b x) k k=0 of special polynomials p k x) = x a) k b x) k For functions of several variables, Bernstein s representation has the form fx 1,, x n ) = c k1k n p k11x 1 ) p knnx n ), k 1k n where p ki ix i ) ef = x i a i ) ki b i x i ) ki In this representation, we store the coefficients c k1 k n in the computer Bernstein polynomials are actively use, eg, in computer graphics an computer-aie esign, where they are not only more computationally efficient, but they also lea within a comparable computation time to smoother an more stable escriptions than traitional computer representations of polynomials Comment A representation of a polynomial as a linear combination of Bernstein polynomials is not the only computationally efficient alternative to a more traitional representation of a polynomial as a linear combination of monomials We can also represent a polynomial as a linear combination of polynomials from some orthogonal basis, or use Lagrange an Newton) polynomials that originate from Lagrange interpolation These polynomials are known an have been applie for several centuries There is a lot of experience of using these polynomials, an there are theoretical results that explain their computational avantages In contrast, Bernstein polynomials are less than a hunre years ol Their avantages are not as thoroughly explore as of other families of polynomials, their use is not as well theoretically justifie In short, there is a gap between the amount of known results about other types of polynomials an the much smaller amount of results about Bernstein polynomials In view of empirical usefulness of Bernstein polynomials, it is esirable to narrow this gap, by performing theoretical analysis of these polynomials The main objective of this paper is to contribute to this gap-narrowing Specifically, we provie a new fuzzymotivate justification of the Bernstein polynomials Bernstein polynomials are useful in interval an fuzzy computations Bernstein polynomials are useful in many computational problems In particular, they are useful in interval computations see, eg, [8]), when we are computing the range f[a 1, b 1 ],, [a n, b n ] ef = {fx 1,, x n ) : x 1 [a 1, b 1 ],, x n [a n, b n ]} of a function fx 1,, x n ) over a given box The efficiency of Bernstein polynomials in interval computations is shown in [2], [3], [4], [5], [9], [13] Such interval computations are extremely useful in fuzzy computations see, eg, [6], [12]), when we know fuzzy values of the inputs x 1,, x n ie, the corresponing membership functions µ i x i )), an we nee to fin the corresponing fuzzy value of y = fx 1,, x n ) ie, the membership function µy)) It turns out that for every α, the α-cut yα) = {y : µy) α} is equal to the range of the function fx 1,, x n ) over the corresponing α-cuts x i α) = {x i : µ i x i ) α}: yα) = fx 1 α),, x n α))

3 This is how fuzzy computations are usually performe by performing interval computations over the corresponing α- cuts, for ifferent values α [0, 1] Bernstein polynomials: open problem Empirically, it is known that the Bernstein polynomials are often more computationally efficient than the traitional monomials However, in spite of many efforts to explain this empirical efficiency, no convincing explanation has been foun so far What we o In this paper, we first show that by using fuzzy techniques, we can get a reasonable explanation of why Bernstein polynomials are more efficient than the traitional monomials Then, we show how this informal explanation can be transforme into a precise mathematical justification II BERNSTEIN POLYNOMIALS: FUZZY EXPLANATION Preliminary step: reucing all intervals to the interval [0, 1] We want to use fuzzy logic to analyze polynomial an piece-wise polynomial approximations In fuzzy logic, traitionally, possible truth values form an interval [0, 1] In some intelligent systems, other intervals are use eg, in the historically first expert system MYCIN the interval [ 1, 1] was use to escribe possible egrees of confience It is well known that it oes not matter much what interval we use since we can easily reuce values x from an interval [a, b] to values t from the interval [0, 1] by taking t = x a ; vice versa, b a once we know the new value t, we can easily reconstruct the original value x as x = a + t b a) To facilitate the use of traitional fuzzy techniques, let us therefore reuce all the intervals [a i, b i ] to the interval [0, 1] In other wors, instea of the original function fx 1,, x n ) : [a 1, b 1 ] [a n, b n ] IR, we consier a new function which is efine as F t 1,, t n ) : [0, 1] n IR, F t 1,, t n ) = fa 1 + t 1 b 1 a 1 ),, a n + t n b n a n )) Vice versa, if we fin a goo approximation F t 1,, t n ) to the new function F t 1,, t n ), we can easily generate an approximation fx 1,, x n ) to the original function fx 1,, x n ) as follows: fx 1,, x n ) = F x1 a 1 b 1 a 1,, x n a n b n a n ) Fuzzy-base function approximations: reminer Fuzzy techniques have been actively use to approximate functional epenencies: namely, such epenencies are approximate by fuzzy rules; see, eg, [6], [7], [12] The simplest case is when each rule has a fuzzy conition an a crisp conclusion, ie, has the type if x is P, then y = c, where P is a fuzzy property such as small ) characterize by a membership function µx), an c is a real number For the case of several inputs, we have rules of the type if x 1 is P 1, x 2 is P 2,, an x n is P n, then y = c The egree to which a given input x i satisfies the property P i is equal to µ i x i ), where µ i x) is the membership function corresponing to the property P i The egree to which the tuple x 1,, x n ) satisfies the conition of the rule ie, the statement x 1 is P 1, x 2 is P 2,, an x n is P n is therefore equal to f & µ 1 x 1 ),, µ n x n )), where f & is an appropriate t-norm an -operation) One of the simplest t-norms is the prouct f & a, b) = a b For this t-norm, the egree to which the above rule is satisfie is equal to the prouct µ 1 x 1 ) µ n x n ) of the corresponing membership egrees When we have several rules, then we get ifferent conclusions c 1,, c r with egrees 1,, r ; we nee to come up with a single values that combines these conclusions The larger the egree i, the more weight we shoul give to the conclusion c i A natural way is thus simply to take the weighte average c c r r This weighte average can be interprete in fuzzy terms if we interpret the combination as the following statement: either the conition for the 1st rule is satisfie an its conclusion is satisfie) or the conition for the 2n rule is satisfie an its conclusion is satisfie) or or the conition for the r-th rule is satisfie an its conclusion is satisfie), where we escribe an as multiplication an or as aition Resulting interpretation of the usual polynomial representation The functions of one variable, the traitional computer representations of a polynomial has the form c 0 + c 1 x + c 2 x c m x m The corresponing approximation can be interprete as the following set of fuzzy rules: c 0 with no conition); if x, then c 1 ; if x 2, then c 2 ; if x m, then c m In fuzzy logic, if we take x as the egree to which x [0, 1] is large, then: x 2 is usually interprete as very large, x 4 = x 2 ) 2 is interprete as very very large, x 8 = x 4 ) 2 = x 2 ) 2 ) 2 is interprete as very very very large, etc, an intermeiate powers x 3, x 5, x 7, etc, are interprete as as some intermeiate heges

4 Thus, the above rules have the form: c 0 ; if x is large, then c 1 ; if x is very large, then c 2, etc Similarly, for polynomials of several variables, we have as many rules as there are monomials c k1 k n x k 1 1 xk n n For example, a monomial c 012 x 0 1 x 1 1 x 2 2 = c 012 x 2 x 2 3 correspons to the following rule: if x 2 is large an x 3 is very large, then c 012 Fuzzy interpretation reveals limitations of the traitional computer representation of polynomials From the fuzzy viewpoint, there are two limitations to this interpretation The first limitation is relate to the fact that an accurate representation requires polynomials of higher egrees, with several istinct coefficients corresponing to ifferent heges such as very, very very, etc In practice, we humans can only meaningfully istinguish between a small number of heges, an this limits the possibility of meaningfully obtaining such rules from experts The secon limitation is that for the purposes of computational efficiency, it is esirable to have a computer representation in which as few terms as possible are neee to represent each function This can be achieve if in some important cases, some of the coefficients in the corresponing computer representation are close to 0 an can, therefore, be safely ignore For the above fuzzy representation, all the terms are meaningful, an there seems to be no reason why some of these terms can be ignore How can we overcome limitations of the traitional computer representation: fuzzy analysis of the problem From the fuzzy viewpoint, the traitional computer representation of polynomials correspons to taking into account the opinions of a single expert Theoretically, we can achieve high accuracy this way if we have an expert who can meaningfully istinguish between large, very large, very very large, etc However, most experts are not very goo in such a istinction A typical expert is at his or her best when this expert istinguishes between large an not large, any more complex istinctions are much harer Since we cannot get a goo approximation by using a single expert, why not use multiple experts? In this case, there is no nee to force an expert into making a ifficult istinction between very large an very very large So, we can as well use each expert where each expert is the strongest: by requiring each expert to istinguish between large an very large In this setting, once we have experts, for each variable x i, we have the following options: The first option is when all experts believe that x i is large: the 1st expert believes that x is large, the 2n believes that x i is large, etc Since we have ecie to use prouct for representing an, the egree to which this conition is satisfie is equal to x i x i = x i Another option is when 1 experts believe that x i is large, an the remaining expert believes that x is not large The corresponing egree is equal to x 1 i 1 x i ) In general, we can have k i experts believing that believing that x i is large an k i experts believing that x is not large The corresponing egree is equal to x k i i 1 x i ) k i For this variable, general weighte combinations of such rules lea to polynomials of the type c ki x k i i 1 x i ) k i, ie, k i to Bernstein polynomials of one variable For several variables, we have the egree p kiix i ) = x ki i 1 x i ) k i with which each variable x i satisfies the corresponing conition Hence, the egree to which all n variables satisfy the corresponing conition is equal to the prouct p k11x 1 ) p knnx n ) of these egrees Thus, the corresponing fuzzy rules lea to polynomials of the type c k1k n p k11x 1 ) p knnx n ), k 1,,k n ie, to Bernstein polynomials So, we inee get a fuzzy explanations for the emergence of Bernstein polynomials Why Bernstein polynomials are more computationally efficient: a fuzzy explanation Let us show that the above explanation of the Bernstein polynomials leas to the esire explanation of why the Bernstein polynomials are more computationally efficient than the traitional computer representation of polynomials Inee, the traitional polynomials correspon to rules in which conitions are x is large, x is very large, x is very very large, etc It may be ifficult to istinguish between these terms, but there is no reason to conclue that some of the corresponing terms become small In contrast, each term x ki i 1 x i ) k i from a Bernstein polynomial, with the only exception of cases k i = 0 an k i = D, correspons to the conition of the type x i is large very large, etc) an x i is not large very not large, etc) While in fuzzy logic, such a combination is possible, there are important cases when this value is close to 0 namely, in the practically important cases when we are either confient that x i is large or we are confient that x i is not large In these cases, the corresponing terms can be safely ignore, an thus, computations become inee more efficient III FROM FUZZY EXPLANATION TO A MORE PRECISE EXPLANATION Main iea Let us show that terms x k i i 1 x i ) k i corresponing to k i 0, ) are inee smaller an thus, some of them can inee be safely ignore To prove this fact, let us pick a threshol ε > 0 ε 1) an in each computer representation of polynomials, let us only keep the terms for which the largest possible value of this term oes not excee ε

5 Traitional computer representation of polynomials: analysis In the traitional representation, the terms are of the type x k1 1 xk n n When x i [0, 1], each such term is a prouct of the corresponing terms x k i i The resulting non-negative function is increasing in all its variables, an thus, its largest possible value is attaine when all the variables x i attain their largest possible value 1 The corresponing largest value is equal to 1 k1 1 kn = 1 Since the largest value of each term is 1, an 1 is larger than the threshol ε, all the terms will be retaine If we restrict ourselves to terms of orer for each variable x i, we get: + 1 possible terms for one variable: n + 1) 2 terms x k1, an x 0 i = 1, x 1 i = x, x 2 i,, x i, 1 xk2 2 for two variables, + 1) n terms in the general case of n variables This number of retaine terms grows exponentially with the number of variables n Bernstein polynomials: analysis For Bernstein polynomials, each term has the prouct form p k1 1x 1 ) p kn nx n ), where p kiix i ) = x ki i 1 x i ) ki The prouct of nonnegative numbers p ki ix i ) is a monotonic function of its factors Thus, its maximum is attaine when each of the factors p ki ix i ) = x k i i 1 x i ) k i is the largest possible Differentiating this expression with respect to x i, taking into account that the erivative of fx) = x k is equal to k x fx), an equating the resulting erivative to 0, we conclue that k i x i p ki ix i ) k i 1 x i p ki ix i ) = 0, ie, that k i = k i Multiplying both sies of this equality x i 1 x i by the common enominator of the two fractions, we get k i 1 x i ) = k i ) x i, ie, k i k i x i = x i k i x i Aing k i x i to both sies of this equation, we get k i = x i hence x i = k i Thus, the largest value of this term is equal to ) ki x ki i 1 x i ) k ki i = 1 k ) ki i This value is the largest for k i = 0 an k i =, when the corresponing maximum is equal to 1; as a function of k i, it first ecreases an then increases again So, if we want to consier values for which this term is large enough, we have to consier value k i which are close to 0 ie, k i ) or close to ie, k i ) For values k i which are close to 0, we have 1 k i ) ki 1 k i ) It is known that for large, this value is asymptotically equal to exp k i ) Thus, the logarithm ) ki ki of the corresponing maximum 1 k ) ki i is ) ki ki asymptotically equal to the logarithm of exp k i ), ie, to k i ln) lnk i ) + 1) Since we have k i, we get lnk i ) ln) an therefore, the esire logarithm is asymptotically equal to k i ln) For the values k i, we can get a similar asymptotic expression k i ) ln) Both expressions can be escribe as i ln), where i enotes mink i, k i ), ie, i = k i when k i, an i = k i when k i We want to fin all the tuples k 1,, k n ) for which the prouct of the terms p ki ix i ) corresponing to iniviual variables is larger than or equal to ε The logarithm of the prouct is equal to the sum of the logarithms, so the logarithm if the prouct is asymptotically equal to n i ln) Thus, the conition that the prouct is larger than or equal to ε is asymptotically equivalent to the inequality n ln) lnε), ie, to the inequality n i C ef = lnε) ln) The number of tuples of non-negative integers i that satisfy the inequality n i C can be easily foun from combinatorics Namely, we can escribe each such tuple if we start with C zeros an then place ones: we place the first one after 1 zeros, we place the secon one after 2 zeros following the first one, etc As a result, we get a sequence of C + n symbols of which C are zeros Vice versa, if we have a sequence of C +n symbols of which C are zeros an thus, n are ones), we can take: as 1 the number of 0s before the first one, as 2 the number of 0s between the first an the secon ones, etc Thus, the total number of such tuples is equal to the number of ways that we can place C zeros in a sequence of C + n symbols, ie, equal to ) C + n = C n + C) n + C 1) n + 1) 1 2 C When n is large, this number is asymptotically equal to const n C Each value i correspons to two ifferent values k i : the value k i = i an the value k i = k i

6 Thus, to each tuple 1,, n ), there correspon 2 n ifferent tuples k 1,, k n ) So, the total number of retaine tuples k 1,, k n ) ie, tuples for which the largest value of the corresponing term is ε is asymptotically equal to 2 n n C Conclusion: Bernstein polynomials are more efficient than monomials As we have shown: In the traitional computer representation of a polynomial of egree in each of the variables, we nee asymptotically + 1) n terms For Bernstein polynomials, we nee 2 n n C terms For large n, the factor n C grows much slower than the exponential term 2 n, an 2 n +1) n Thus, in the Bernstein representation of a polynomial, we inee nee much fewer terms than in the traitional computer representation an therefore, Bernstein polynomials are inee more efficient Comment There exist other explanations of why Bernstein polynomials are more computationally efficient; see, eg, [10] an references therein; the main avantage of our explanation is that it comes from fuzzy analysis an is, therefore, intuitively clearer than previously known explanations IV CONCLUSIONS In many practical problems, we approximate a epenence y = fx 1,, x n ) of a quantity y on the quantities x 1,, x n by polynomials or by piece-wise polynomial functions splines) The accuracy an computational efficiency of the resulting approximation epens on how we represent the corresponing polynomials A polynomial is usually efine as a linear composition of monomials n n n fx 1,, x n ) = c 0 + c i x i + c ij x i x j + j=1 Because of this, the traitional way to represent a polynomial in a computer is to keep the corresponing coefficients c 0, c 1,, c n, c 11,, c 1n,, c n1,, c nn, In many cases, however, it is more computationally efficient, for each box [a 1, b 1 ] [a n, b n ], to represent polynomials on this box as linear combinations of so-calle Bernstein polynomials x 1 a 1 ) k1 b 1 x 1 ) k1 x n a n ) kn b n x n ) k n While empirically, this representation is often more efficient, there has been no intuitively convincing theoretical explanations for this efficiency In this paper, we use fuzzy logic to provie an intuitive theoretical explanation of why Bernstein polynomials are more efficient than the traitional monomial representations of polynomials Specifically, we first provie a fuzzy interpretation of Bernstein polynomials, an then we transform this intuitive interpretation into the precise justification of the relative efficiency of Bernstein polynomials ACKNOWLEDGMENT This work was supporte in part by the National Science Founation grants HRD Cyber-ShARE Center of Excellence) an DUE , an by Grant 1 T36 GM from the National Institutes of Health The authors are thankful to anonymous referees for valuable suggestions REFERENCES [1] R P Feynman, R B Leighton, an M Sans, Feynman Lectures on Physics, Aison-Wesley, Boston, Massachusetts, 2005 [2] J Garloff, The Bernstein algorithm, Interval Computation, 1993, Vol 2, pp [3] J Garloff, The Bernstein expansion an its applications, Journal of the American Romanian Acaemy, 2003, Vol 25 27, pp [4] J Garloff an B Graf, Solving strict polynomial inequalities by Bernstein expansion, In N Munro, eitor, The Use of Symbolic Methos in Control System Analysis an Design, volume 56 of IEE Contr Eng, Lonon, 1999, pp [5] J Garloff an A P Smith, Solution of systems of polynomial equations by using Bernstein polynomials, In: G Alefel, J Rohn, S Rump, an T Yamamoto es), Symbolic Algebraic Methos an Verification Methos: Theory an Application, Springer-Verlag, Wien, 2001, pp [6] G J Klir an B Yuan, Fuzzy Sets an Fuzzy Logic, Prentice Hall, Upper Sale River, New Jersey, 1995 [7] V Kreinovich, G C Mouzouris, an H T Nguyen, Fuzzy rule base moeling as a universal approximation tool, In: H T Nguyen an M Sugeno es), Fuzzy Systems: Moeling an Control, Kluwer, Boston, Massachusetts, 1998, pp [8] R E Moore, R B Kearfott, an M J Clou, Introuction to Interval Analysis, SIAM Press, Philaelphia, Pennsylvania, 2009 [9] P S V Nataraj an M Arounassalame, A new subivision algorithm for the Bernstein polynomial approach to global optimization, International Journal of Automation an Computing, 2007, Vol 4, pp [10] J Nava an V Kreinovich, Theoretical Explanation of Bernstein Polynomials Efficiency: They Are Optimal Combination of Optimal Enpoint- Relate Functions, University of Texas at El Paso, Department of Computer Science, Technical Report UTEP-CS-11-37, July 2011, available as [11] H T Nguyen an V Kreinovich, Applications of continuous mathematics to computer science, Kluwer, Dorrecht, 1997 [12] H T Nguyen an E A Walker, First Course In Fuzzy Logic, CRC Press, Boca Raton, Floria, 2006 [13] S Ray an P S V Nataraj, A New Strategy For Selecting Subivision Point In The Bernstein Approach To Polynomial Optimization, Reliable Computing, 2010, Vol 14, pp [14] L A Zaeh, Fuzzy sets, Information an control, 1965, Vol 8, pp