Polynomial Inclusion Functions

Transcription

1 Polynomial Inclusion Functions E. e Weert, E. van Kampen, Q. P. Chu, an J. A. Muler Delft University of Technology, Faculty of Aerospace Engineering, Control an Simulation Division E.eWeert@TUDelft.nl Abstract When using interval analysis, the bouns of an inclusion function are often non-tight ue to epenency effects. The benefit of Taylor Moels (TMs) or Verifie Taylor Series (VTSs) is the use of higher orer erivatives terms, significantly reucing the epenency effect. In this paper, it is assume that the require information to erive these inclusion functions is obtaine using automatic ifferentiation. The rawback of TMs an VTSs is that not all available information is use, resulting in non-optimal inclusion functions. In this paper the Polynomial Inclusion Function (PIF) is presente, which is guarantee to form equal or shaper enclosures than any (combination of) Taylor Moel(s) efine using the same set of information. The PIF is erive for the one imensional case. Etension to n-imensional functions is performe via application of the PIF to every imension inepenently. The performance of the PIF is compare to that of Verifie Taylor Series for multiple (non-linear) functions an is shown to yiel to superior inclusions. Moreover, unlike with TMs or VTSs, increasing the orer of the PIF will always sharpen its bouns. Keywors: Polynomial inclusion function, interval analysis, Taylor Moels, Verifies Taylor Series, inclusion function AMS subject classifications: 5- Introuction For any global optimization problem it is crucial to fin guarantee bouns, i.e. an inclusion function, on a given (usually non-linear) function f whose output epens on the variable parameters for which it is optimize. With these bouns one can fin all global/local solutions of the given problem using simple branch an boun methos. For all optimization problems, it hols that the computational loa neee to solve the problem highly epens on the sharpness of the obtaine bouns. The sharper the bouns, the faster regions of the search space can be iscare, leaing to less computational effort (unless the cost of obtaining sharper bouns is too high). Fining Submitte: April, ; Revise: July,, October 9,, September 8,, an November 8, ; Accepte: November 3,. 83

2 8 e Weert et al, Polynomial Inclusion Functions sharper bouns at low cost is therefore a crucial part of solving optimization problems. Most inclusion functions escribe the bouns on a function as a polynomial. When using stanar interval arithmetic one obtains a zero-orer polynomial escribing the upper boun an one for the lower boun. In the centere forms [, 7, 8] the obtaine inclusion is a set of two first orer polynomials for the upper boun an another set for the lower boun. Higher orer polynomial inclusion functions can be obtaine using Taylor series theory. One of such inclusion function is the Taylor Moel. In this paper it is assume that the set of information require to construct these type of inclusion functions is obtaine via automatic ifferentiation in combination with interval arithmetic. All claims in this paper are base on this assumption. Taylor moels have been evelope by Lanfor aroun 98, subsequently stuie by Eckmann, Koch, Wittwer, Berz, Makino an Hoefkens [7,, 3, ], an have been applie successfully to many problems [, ]. Taylor Moels are shown to have ecellent performance in situations where the omain is small an are often use to remove the wrapping effect encountere when performing guarantee integration. For larger omains the orer of the polynomial must increase to prevent a remainer blow-up. For optimization problems in which the epenency effects increase when taking a higher orer erivative of the function [5] the remainer blow-up can only be prevente by reucing the omain with (e.g., bisecting the omain). Although wiely applie in the reliable computing community an applicable in n imensional problems, the Taylor Moels are not always optimal, an other inclusion functions can result in tighter bouns [5]. Generally speaking the optimal form of the inclusion function is problem epenent. The main focus of this paper is on eriving a generic inclusion function which uses all available information as efficiently as possible to form the sharpest possible bouns. In case of Taylor Moels one nees to erive, over the entire omain [], the natural inclusion function of the n th orer erivative ([ n f([])) an the to (n ) th orer erivative value at the epansion point. As previously state, the assumption is that the erivative information is erive using automatic ifferentiation techniques. The key aspect is that when the n th orer erivative information is erive one automatically obtains information regaring the erivatives of lower orer ( to (n )). This means that the entire information set S, for a one-imensional function f, is given by: { [ ]} i f S : j f i,, i =,,..., n, j =,,..., n +. () = j =[] In the current application, the Taylor Moels (or Taylor series) only use the information obtaine from [ n f([])] leaving a lot of available information unuse. The first contribution of this paper is the erivation of a new inclusion function calle the polynomial inclusion function (PIF) which provies a guarantee (equal or) sharper inclusion than any (combination of) Taylor Moel(s) erive using the same set of information. The PIF is an inclusion function base on a set of piecewise polynomials escribing the guarantee upper (P f ) an lower boun (P f ) on function f for omain The efinition of small is problem epenent. Automatic ifferentiation packages recor a irecte acyclic graph (DAG) while computing the function value [8]. When the n th orer erivative is compute, the DAG automatically contains the information of the to (n ) th orer erivative.

3 Reliable Computing, 85 []: f () [ P f (), P f () ] []. () This paper focuses on the theoretical evelopment of the new inclusion function. The goal is to show information can be use more efficiently than for Taylor Moels. Since the computational loa is an important aspect of any optimization algorithm, a cost analysis is performe. The secon contribution of this paper is the introuction of a computational light version of the propose PIF to show that faster optimization is possible than when using Taylor Moels. In this paper the one-imensional case is escribe. The PIF can be etene to multiple imensions by applying it to each imension separately as will be emonstrate in section. The one-imensional case frequently is encountere in ynamic optimization problems, e.g., trajectory optimization, where the cost function commonly inclues an integral [] over time: t f J = Φ ( (t ), t, (t f ), t f ) + t L ( (t), u (t), t) t, (3) where Φ is a function enoting the penalty for en-point constraints, an L is the Lagrangian. By having a sharp inclusion of the Lagrangian (a generally non-linear function) in polynomial form, the integral can be compute easily. Another large fiel of research is that of bouning the solutions of ODEs (require in for instance initial value problems) for which Taylor Moels frequently are use [9][](Packages COSY-INFINITY [3], VSPODE [9], an VNODE-LP [5] all incorporate some form of Taylor Moels). The propose inclusion function can be applie in this fiel as well. In section, a brief review of guarantee Taylor series epansions is given. The effect of Taylor series orer on the sharpness of the inclusion function will be clearly emonstrate. In section 3 the erivation of the PIF will be performe in steps. With each step the information content of the PIF will be increase resulting in guarantee sharper bouns (over the entire omain) compare to the previous step. The erive PIF will be valiate an compare to Verifie Taylor Series (introuce in the net section) for several eamples in section. A cost analysis is performe in section 5, an the etension of the propose metho to higher imensional function is given in section. Finally, in section 7, conclusions an recommenations for further work are given. Verifie Taylor Series (VTS) Any (non-linear) function f can be approimate by a Taylor series. By using interval analysis one can efine an inclusion function base on a Taylor series epansion which provies guarantee bouns on f for a given omain. Taylor s formula with remainer for a one imensional function is given by: f () = n v= f (v) ( ) v! ( ) v + g () g () f (n+) (ξ) ( ξ) n, () g () (ξ) n!

4 8 e Weert et al, Polynomial Inclusion Functions where g is an arbitrary function with non vanishing erivative strictly between an, ξ lie strictly between an, an f (v) = v f()/ v3. The form of the remainer epens on the choice of the function g. When choosing g (y) = ( y) n+ g () (y) = (n + ) ( y) n, (5) where y is the inepenent variable, one obtains the Lagrange s Remainer: L n f, (ξ, ) = f (n+) (ξ) (n + )! ( ) n+, () where again ξ lies strictly between an. For the multivariate case we can use the following notation for the n-th orer Taylor polynomial an corresponing remainer: { Tf, n () = i +...+i f() (,) } i... (,) i i +...+i n i... i i!... i! { } L n (7) f, (ξ, ) = n+ f(ξ) i +...+i =n+ i... i (,) i... (,) i i!... i!. Forming a Guarantee Inclusion Function The bouns on the remainer can be etermine by substituting the interval [] for all an ξ in the epression for L n f,. By using interval arithmetic rigorous bouns are obtaine. This approach results in a single interval for the remainer I: I = L n f, ([], []) = i +...+i =n+ { n+ f([]) i... i ([ ],) i... ([ ],) i i!... i! The combination of (T, I) is calle a Taylor Moel []. The with of I epens on the with of [] an on the function f(). If the omain [] is large then the multiplication (,) i... (,) i i!... i causes a blow-up of the remainer if the (n + )-th orer! erivative of f is non-zero. Another ifficulty is the number of occurrences of in the (n + )-th orer erivative of f. As for any function multiple occurrences of can cause overestimation, which yiels large remainer bouns [5]. As an alternative metho one can use Taylor Moel arithmetic from Makino an Berz to erive the remainer. A trae-off between accuracy an spee must be mae to ecie which metho to use. In this paper the metho of eriving the Taylor coefficients is use. What is important to realize is that the remainer also can be kept as a function of. Taylor series theory states that a guarantee inclusion can be forme when inserting [] for only ξ: I() = L n f, ([], ) = i +...+i =n+ } { n+ f([]) i... i (,) i... (,) i i!... i! The resulting inclusion function, i.e. (T, I()), is calle a Verifie Taylor Series (VTS) in this paper. From the efinition of the VTS an TM one can erive the following: 3 For the work in this paper the function f may be analytically or numerically ifferentiable. }... (8) (9)

5 Reliable Computing, 87 f = cos(π) TM VTS f Figure : Inclusion of f = cos(π), [, ] using the Taylor Moel an Verifie Taylor Series. The epansion point for both inclusion functions is =.5, an the orer is 3. there eists a parameter δ an a parameter δ such that the following hols : V T S() T M(), [δ, δ ]. () Uner the assumption that automatic ifferentiation is use to erive the require information for construction of the VTS an TM the values of δ an δ can be set to inf[] an sup[] respectively. This means that the following hols: V T S() T M(), []. The ifference between the TM an VTS is emonstrate in Figure. Since the VTS yiels sharper inclusions it is use as a reference in the remainer of this paper.. Use of Derivative Information The n th orer VTS is forme using the natural inclusion function of the (n+) th orer erivatives. As state in the introuction, it is assume that the erivative natural inclusion function [f (n+) ([])] is erive using automatic ifferentiation techniques such that all other erivatives up to orer (n + ) are reaily available. Since only [f (n+) ([])] is use for the construction of the TM/VTS, a lot of valuable information is isregare. In Figure VTSs up to orer for function f = cos(π), [, 3] are given. As one can clearly see, the VTSs of orer > all violate the bouns of the [f([])] (equal to V T S f ()) at some point []. Moreover, the bouns of other erivative inclusion functions may also be violate, i.e., upper boun violation: [ i V T S sup i ] [ i f > sup i [] ], [], () This results is inepenent of the way the remainer of the TM is erive, e.g. automatic ifferentiation or TM arithmetic.

6 88 e Weert et al, Polynomial Inclusion Functions.5.5 f() VTS f, () VTS f, () f VTS f, () VTS f, () VTS 3 f, () VTS f, () Figure : Taylor Moels for function f = cos(π) for omain X = [, 3] an epansion point =.5. (same hols for lower boun violations). Since the bouns of [f (i) ([])] are guarantee, one can use this information to improve the inclusion. In the following section a metho is given that makes sure that the VTS will not violate any of the bouns set by the erivative inclusion functions. 3 Polynomial Inclusion Function (PIF) The iscussion on the Verifie Taylor Series showe that not all available information regaring erivative inclusions is use. In this section the polynomial inclusion function (PIF) is erive that uses the available erivative bouns more efficiently. The resulting PIF provies a guarantee equal or sharper enclosure than any VTS constructe base on the same set of available information. The PIF consists of two piecewise polynomials, one bouning the function from above an one from below: [ ] f () P IFf () = Pf (), P f () [], () where f enotes the function an the maimal erivative orer of which information is use in the construction of the PIF. The PIF erive in this paper is base on Taylor series epansion theory. To simplify the iscussion the erivation of the PIF starts by looking at the upper boun on function f for. All other remaining bouns (upper boun an lower boun for, an lower boun for ) can be erive using the same proceure after having performe a simple coorinate mapping. The results given in this section show the PIF for the entire omain to emonstrate the overall performance. The eample of f = cos(π), =.5, [, 3] given in Figure is use to eplain the consequences of each step in the erivation. With each step, the bouns of the PIF become more

7 Reliable Computing, 89 tight or remain equal over the entire omain, i.e., for every [] the bouns of the new PIF are equal to or within the bouns of the PIF of the previous step. 3. Combining VTS Proposition 3.. Given a certain set S containing erivative inclusion functions for omain [] an erivative evaluations at location for function f, { [ i f S : j f i, = j =[] ]}, i =,,..., n, j =,,..., n +, [], (3) it is guarantee that the optimal inclusion function has sharper or equal bouns compare to the bouns of VTS of any egree, which has been erive using the same set of information, for the entire omain []. Proof. Since the VTS of any given egree is a guarantee inclusion function of function f: f () V T S f, (), n, [], () { one can select the sharpest bouns at each [] provie by one of the VTS. The resulting boun is guarantee to correctly boun the function f from above an below: { [ ]} [ ]} ma inf V T Sf, () f () min sup V T Sf, () n, []. (5) Since all VTSs are erive using the given erivative information, the optimal inclusion function base on the same information must yiel sharper or equal bouns. Proposition 3. can be use to erive a PIF. To erive the piecewise polynomial P, f one must etermine the intersection points between each pair of VTSs to etermine the lowest value VTS upper boun for each location in []. Fining the intersection points can be formulate as a root fining problem: h() = sup V T S z () sup V T S y () =, for which many methos are available. For polynomials up to orer a close form solution eists [] (such as the quaratic formula for orer ). In the work of Mekwi [] the methos of Bairstow, Bernoulli, Graeffe, Müller, Newton-Raphson, Jenkins- Traub an Laguerre are eplaine. Most available methos provie numeric approimations for the roots, an others use interval analysis to rigorously boun all roots. Irrespectively of the applie metho for fining the roots, the PIF will remain a guarantee inclusion of the function if the omain switch point is chosen such that inf V T Sf, z ( ) sup V T S y f, ( ) when transforming from egree z to y. For the given eample of f = cos(π) the resulting PIF is represente in Figure 3. Note that the sharpest boun is not necessarily forme by the highest orer VTS as can be seen from the P IFf, : P f = sup V T S f, (), [.,.595] sup V T Sf, (), [.595,.53] sup V T Sf, (), [.53,.978] sup V T S f, (), [.978,.] sup V T Sf, (), [.,.75] sup V T S f, (), [.75, 3.] ()

8 9 e Weert et al, Polynomial Inclusion Functions.5.5 f() VTS f, () VTS f, () f VTS f, () VTS f, () VTS 3 f, () VTS f, () PIF f () Figure 3: Verifie Taylor Series (VTSs) an Polynomial Inclusion Function (PIF) for function f = cos(π) for omain X = [, 3] an epansion point =.5. The PIF is forme by applying Proposition 3.. Pf = inf V T S f, (), [.,.33] inf V T Sf, (), [.33,.9] inf V T S f, (), [.9,.8] inf V T Sf, (), [.8,.53] inf V T Sf, (), [.53,.] inf V T S f, (), [., 3.]. (7) 3. Lowering VTS Orer The PIF forme using Proposition 3. always provies equal or sharper bouns than any VTS. However, the information regaring the guarantee erivatives bouns is not fully use, i.e., the PIF may still violate these bouns. Proposition 3. can be use to form a PIF that, per omain, will consist of a polynomial that oes not violate any erivative bouns (up to the orer of the polynomial). Proposition 3.. Consier a polynomial P (): efine on omain [] for which i f() i Then the polynomial { q P q () = j! j= P () = + j= j! ai ( ))j, (8) i P () i, i [, + ], []. (9) i P () i ( ) j } + (q + )! b ( )q+, ()

9 Reliable Computing, 9 where b q+ f/ q+, [], will also boun the function f an its erivatives up to orer (q + ) from above on omain [, sup[]]. Proof. Since the value b is always equal or larger than the (q + ) th orer erivative of f in omain [] the following hols q P q () q q P q () q Since it is also guarantee that i P q () i q f() q i f() i q f() q [, sup[]]. (), i [, q], () (irectly erive from the properties of P ()), the following must hol: i P q () i This completes the proof. i f() i, i [, q + ], [, sup[]]. (3) The bouning of function f from above is guarantee for any [] when a VTS is use. By efinition, the bouns obtaine for the i th orer erivative of f via the VTS are guarantee: [ ] i f i V T S f i [], i +. () i This means that when one uses the upper boun of a VTS of orer to efine the polynomial P () an using b = sup[f (q+) ([])], all conitions of Proposition 3. are satisfie. Proposition 3. can be use to efine a Taylor polynomial of orer q which will always boun the function f from above (an its erivatives up to orer q + ). The new Taylor polynomial again satisfies the conitions of the proposition. This means that one can keep on switching to lower orer Taylor polynomials when esire without sacrificing the guarantee of bouning function f from above. Proposition 3. can be use to efine a PIF which has Taylor polynomials for each omain in which o not violate any of the erivative bouns (up to the orer of the active polynomial (q)). The iea is that the orer of the Taylor polynomial in a given sub-omain (e.g., one part of the PIF forme by Proposition 3.) is reuce to q if the (q + ) th orer erivative of the polynomial crosse the sup[f (q+) ([])] boun. By altering the polynomial to a lower orer, the inclusion function is guarantee not to violate that erivative boun. As an eample consier the function f = cos(π) again for VTS up to orer. When Proposition 3. an Proposition 3. are applie a PIF can be erive which has guarantee equal or sharper bouns than any VTS. The result is represente in Figure, an the PIF escription is given below (inication of the polynomial orer use per sub-omain of ):

10 9 e Weert et al, Polynomial Inclusion Functions.5 f.5 f() VTS f, () VTS f, () VTS f, () VTS f, () VTS 3 f, () VTS f, () PIF f () Switch point Figure : Verifie Taylor Series(VTSs) an Polynomial Inclusion Function (PIF) for function f = cos(π) for omain X = [, 3] an epansion point =.5. The PIF is forme by applying proposition 3. an 3.. At a switch point the polynomial of the PIF changes orer. P f egree of polynomial [.,.] [.,.7] [.7,.8] [.8,.5] 3 [.5,.88] 5 [.88,.97] [.97,.39] [.39,.37] 5 [.37,.7] 3 [.7, 3.] P f egree of polynomial [.,.333] [.333,.83] 3 [.83,.9] 5 [.9,.] [.,.8] [.8,.5] 5 [.5,.88] 3 [.88,.9] [.9,.7] [.7, 3.] Comparing to the result given in Figure 3, one can see that the effect of Proposition 3. can be severe. The obtaine inclusion is much sharper than the original VTSs base on the same set of information. 3.3 Transitions Between Taylor Polynomials Although the erive PIF is alreay an improvement, there still remains one issue: the transition between Taylor polynomials in the PIF. As an eample, consier the two zoom plots of Figure shown in Figure 5. Clearly the transition in both plots is iscontinuous for erivative orer higher than. A jump in the n th orer erivative means that the (n + ) th orer erivative oes not eist. The latter is not possible for the unerlying function f since it is guarantee that the erivatives remain within the

11 Reliable Computing, 93 f f() VTS f, () VTS f, () VTS f, () VTS f, () VTS 3 f, () f VTS f, () PIF f () Switch point Figure 5: Zoom plots of the result given in figure. The figure emonstrates the iscontinuous behavior of the PIF in the first orer erivative between polynomials. bouns specifie by the natural inclusion function [f (n) ([])]. This information can be use to sharpen the bouns on f as is emonstrate net. Consier a location for which the function f is at its maimum. Per efinition the first orer erivative of f must be zero, an the secon orer erivative. Nothing can be sai of the values of the higher orer erivatives since it epens on the value of the secon orer erivative (e.g., if it is zero then the 3 r orer erivative must be an so on). The bouns on f given by [f([])] are guarantee thus for any point on this boun the previous conitions of the first an secon orer erivative must hol. Now consier the PIF represente in Figures an 5 at location =. where the transition is mae from a zero orer Taylor polynomial to a first orer Taylor polynomial. Suppose that a point [.,.] eists where f is equal to the active Taylor polynomial P, an a point f [.,.7] eists where f is equal to the active Taylor polynomial P f on that omain (see Figure ). If a secon orer polynomial is use to efine the transition between the two points then the following must hol: p( ) = P ( f ) p = p () = P ( f ) + a ( ) p( ) = P f ( ) p = p X = a P f = = + = P f a P f a, (5) where a is the constant secon orer erivative value to be etermine. To form a guarantee inclusion of the function f the value of must be as small as possible

12 9 e Weert et al, Polynomial Inclusion Functions f p() Figure : Transition polynomial between Taylor polynomials base on minimal secon orer erivative information. which is obtaine as follows: P f [ ] > a = sup f =[] [ ]. () < a = inf f =[] P f By using the bouns obtaine by the natural inclusion function the transition is guarantee to be a vali upper boun on the function f. The emonstrate principle of transition between Taylor polynomial hols for any transition between Taylor polynomial. However, the proceure for efining the transition polynomial becomes less trivial for higher orer transitions (i.e., p has a higher egree). As an eample, the previously etermine secon orer polynomial still violates the thir orer erivative boun when the transition is mae between p an the Taylor polynomials at an (jump in secon orer erivative) which is not vali. Incluing higher orer erivative information will complicate the erivation, but sharper inclusion functions can be forme. The research on the smooth transitions between polynomials in the PIF is subject of ongoing research. This inclues the analysis of spee versus tightness trae-offs. The main aspects have been highlighte here for completeness but will not be inclue in the net section ealing with performance evaluation of the PIF. Comparing Methos To etermine the possible improvement of the bouns forme by the PIF compare to the stanar Verifie Taylor Series (VTS), several test cases are use:

13 Reliable Computing, 95 f () = cos(π) f () = ep( π ) f 3() = n i= sin(π i! i + π i! ), n = 5 for [, ] (see Figure 7). For all test cases the volume V between the upper an lower bouns right of the epansion point is compute for the VTS up to orer, PIF forme using only Proposition 3. (P IF ), PIF forme by using Proposition 3. an 3. (P IF ). To see the effect of the with of the omain (VTS accuracy increases with a ecrease in omain with []) all test cases are evaluate for several omains with varying iameters right of the epansion point =, i.e., [] = + w([]) [, ] (where w([]) enotes the with of the omain []). The first test function is a simple function to emonstrate the basic properties of the VTS an the PIF. The other two functions are selecte to emonstrate the performance in case of non-vanishing epenency effects, i.e., functions leaing to remainer blowup in case of Taylor Moels. The results for test function f, f, an f 3 are given in Figures 8, 9, an, respectively. The first conclusion is that the VTSs are inee effective for small omains [] ( small is problem epenent). The effect of remainer blowup is clearly visible in all figures. One can see that for a given w([]) the inclusion might become worse with increasing VTS orer before ecreasing again. This behavior is clearly not present for both P IF an P IF. For the PIF the accuracy always increases with increase orer, i.e., increasing available information. Moreover, it is clearly visible that the PIF always outperforms the VTS, in particular for larger omains. Looking at the figures where the performance of P IF is compare to that of P IF one can clearly see that a huge improvement can be realize (> 5%, epresse as a percentage of the V (P IF )) especially in the mile values of w([]). For larger values of w([]) the effect in % rops since the volume V (P IF ) also becomes larger but the in terms of magnitue the gain is increasing. From all figures it becomes clear that the three methos: VTS, P IF, an P IF, perform almost equal for small w([]) an large. This is ue to the fact that the PIF consist of a single polynomial (the optimal VTS) for the largest part of the omain. The reason that the number of switch point, i.e., number of polynomials, in the P IF is not is that at the en of the omain the switch to lower orer polynomials is mae. Since this only happens at the very en of the omain, the increase in efficiency is minimal. The main conclusion is that the performance of the P IF compare to the VTS an P IF increases with increasing function compleity. Due to epenency effects the bouns on the erivatives are wiene leaing to non-tight bouns which affect the performance of the VTSs consierably. To attain the same accuracy as for simpler functions the moel orer must be increase consierably especially for larger omains. The latter is far less severe for the propose PIF. Since higher orer methos are most useful for comple functions (for less comple functions stanar linear methos woul suffice) the introuction of the PIF is a valuable aition to the fiel of inclusion functions.

14 9 e Weert et al, Polynomial Inclusion Functions 3 f f f 3 f Figure 7: Test function use for the comparison of Taylor Moels an Polynomial Inclusion Function performance. 5 Computational Cost Analysis The main message in this paper is that more efficient hanling of available information is possible, i.e., a tighter inclusion function can be obtaine without aitional function evaluations. Although this is the central topic, the overall computational loa require to make the PIF shoul also be consiere. One can argue that the aitional computational loa, require to erive the PIF, coul also have been use to etermine a secon VTS on a sub-omain thereby improving the sharpness of the inclusion. In this section a cost analysis is mae to ientify the cases where it is more beneficial to use the PIF than the VTS an vice versa. The performance of both methos is epresse in terms of obtaine inclusion function sharpness within the same amount of computation time. The metho of eriving the PIF requires the use of a root fining algorithm. The computational loa for solving a one-imensional root fining problem is relatively low but one can argue that the accumulate computational loa can be high. To make a fair comparison between methos, both the PIF an the VTS implementation must be optimize. For the VTS only two function calls must be mae, one at the epansion point an one for the entire omain. All the require information is euce using automatic ifferentiation. The more comple the function, the longer it takes to erive the DAG require for eriving the erivative information. The computational loa of the PIF consists of that of the VTS an aitional loa to apply the propositions given in this paper. To eliminate the nee for a numeric root fining algorithm (for higher orer polynomials), a light version of the PIF construction algorithm is mae: Algorithm given in Table. This algorithm, which is base on Proposition 3. only, was foun

15 Reliable Computing, 97 to yiel similar results as the one using both propositions (compare Figure with Figure ). Note that the guarantee that a tighter enclosure is foun compare to the VTS is lost when only using Proposition The computational loa require for eecuting Algorithm is very low since only first orer polynomial root fining must be performe. Algorithm is be compare to the VTS for the function f 3(). The computational loa of the erivation of the VTS (t V T S) an the computational loa of the require aitional computations to erive the PIF (t P IF ) are shown in Figure (t P IF = t V T S + t P IF ). As one can see, the ae computational loa to erive the PIF is always lower than that require for eriving a VTS (± half for this eample). This means that at most two VTSs can be etermine in the time it takes to erive one PIF. In Figures an 3 the comparison is mae between the inclusion performance in case of using two VTSs (erive for omain [inf[], mi[]] an [mi[], sup[]] respectively - epansion point at the beginning of each omain) an the performance of the PIF erive using Algorithm for the entire omain. Test function f 3 is use with once n = 5 an once n =. The results emonstrate that inee applying two VTSs instea of one yiels better results. This is clearly emonstrate when only the zero orer erivative bouns are use, i.e., =. The performance of the VTS is always better than that of the PIF. From Figures an 3 one can conclue that the performance of the PIF with respect to the VTSs is problem epenent. The more comple the function, the better the performance of the PIF in terms of V (V T S). The same hols for the with of the omain which is investigate. As also the results in the previous section show, it is problem epenent which approach yiels the most optimal overall performance in terms of inclusion bouns for a given amount of computation time. If the omain of interest is small, or the function is smooth, there is no ifference between V (V T S) an V (P IF ) since both will consists of eactly the same polynomial. Therefore the VTS performs better in terms of the computational loa. For larger omains an more non-linear functions the PIF quickly becomes more suitable to use, both in terms of sharpness of the bouns an in terms of computational loa. n-dimensional Case The iscussion of the PIF has been restricte to one-imensional functions. To be easily applicable to higher imensional functions, it is propose to apply the proceure given in this paper to all imensions inepenently. A higher imensional Taylor series is given by: T (n) f, () = i +...+i n +L n f, (ξ, ) { i +...+i f() i... i (,) } i... (,) i i!... i!. (7) 5 In the performe research the enclosure was always foun to be sharper than that of the VTS (see also Figure ).

16 98 e Weert et al, Polynomial Inclusion Functions Table : Algorithm : Determination of P IF f.. Determine DAG for epansion point an omain [] yieling [f ()] for an [].. Determine upper boun P r f for omain [, sup[]] using algorithm a. Apply transformation on y-ais: [f ()] = [f ()], for both an [] b. Determine upper boun P f for omain [, sup[]] using algorithm c. Apply transformation on y-ais on P : P f P rf 3a. Apply transformation on -ais: [f ()] = [f ()], uneven for both an [] 3b. Determine upper boun P f for omain [inf[], ] using algorithm 3c. Apply transformation on y-ais an -ais on P : P f P lf a. Apply transformation on y-ais: [f ()] = [f ()], for both an [] b. Determine upper boun P f for omain [inf[], ] using algorithm c. Apply transformation on -ais on P : P f P l f 5 Create full PIF: P IFf () = [P f, P f ] [] Pf = [P l f, P r f ], P f ] = [P l f, P r f ] Table : Algorithm : Determination of the upper boun piecewise Taylor polynomial P f.. Initialize P f, = sup[v T S f ] with =, set q =, eit = false.. Evaluate sup[v T S f (sup[])] an set ˆ such that: sup[v T S ˆ f (sup[])] sup[v T S f (sup[])] WHILE!eit AND q > ˆ. Determine location where P q f, = sup[f q ] 3. IF <= sup[] THEN compute P q using Proposition 3. ELSE eit f, = true.. Check erivative values: IF i P q f > sup[f i ([])] for any i [, q ˆ] THEN eecute algorithm i 3 to etermine i an. 5. Set = + an compute P i using Proposition 3.. f,. A P q f, as part of the piece-wise polynomial an set q = i, = END

17 Reliable Computing, 99 Table 3: Algorithm 3: Check erivative bouns.. Set i =, i [, q ˆ]. FOR i [, q ˆ]:. Set ɛ = WHILE ɛ <.(sup[] ) / i = + i 3. IF sup[f i+ ([])] THEN set ɛ =, i =. ELSE Compute i = ɛ/ sup[f i+ ([])] END 5. Define i such that i < i, i [, q ˆ] Define = i END. ɛ = sup[f i ([])] i P q f The terms corresponing solely to one imension can be taken out: Tf, n () = { } T (n) f,, ( ) + L n f,, (ξ, ) + R Tf, n, ( ) = { } i i f( ) (,) i n i i! L n f,, (ξ, ) = n+ f (n+)! n+ (, ) n+ ξ, (8) where R contain all the cross terms. By applying the PIF metho to all (T (n) f,, ( ),L n f,, (ξ, )) inepenently, the metho given in this paper can be applie to higher imensions. 7 Conclusions A novel inclusion function has been introuce calle the Polynomial Inclusion Function (PIF). It has been proven an shown through several eamples that the PIF is guarantee to provie equal or sharper bouns that any (combination of) Taylor Moels an Verifie Taylor Series without the nee for aitional information, i.e. function evaluations. The assumption hereby is that the require erivative information is erive using automatic ifferentiation. Irrespectively of the assumption mae, the accuracy of the PIF always improves with increasing orer, a trait that Taylor Moels or Verifie Taylor Series o not possess. This means that the PIF oes not suffer from the inclusion function blowup effect encountere with Taylor Moels (remainer blowup) for highly non-linear functions (non-vanishing erivatives) an/or wier omains. The PIF has been euce for the one-imensional case an can be easily etene to n-imensional functions. In this paper the PIF has been compare to the VTS inclusion function. In future research the PIF will be compare to TMs which have been constructe using TM arithmetic. It has been shown that given a TM an a VTS, there eist a omain [ δ, + δ ], δ, δ for which the VTS (an thus also the PIF) is guarantee to have tighter enclosure of f than the TM, irrespectively of the metho use for

18 3 e Weert et al, Polynomial Inclusion Functions eriving the remainer. The fact that the inclusion forme by a PIF start with nearly zero with is an avantage over TM which have a constant with enclosure (=w(i)) over the entire omain. Future research will inicate what the value of δ, will be for ifferent types of function an ifferent omains. Moreover, a thorough investigation regaring accuracy versus generation cost must be mae to see if the new PIF can be use more efficiently than TM. The TM of Makino an Berz compromise accuracy over spee. If in the time a PIF is erive, multiple TM can be erive, then the overall accuracy of combine TMs might be higher. Although the PIF provies significantly improve bouns compare to VTS, there still remains room for improvement. Future research will focus on incluing even more (available) information in the construction of the PIF to obtain even sharper bouns. The results presente in this paper prove that the PIF are a worthy aition to the fiel of inclusion functions. Acknowlegements This work is performe as part of the MicroNED MISAT cluster, project -D-. References [] R G Ayoub. Paolo Ruffini s contributions to the quintic. Archive for History of Eact Science, 3:53 77, 98. [] M. Berz an G. Hoffstatter. Computation an application of Taylor polynomials with interval remainer bouns. Reliable Computing, :83 97, 998. [3] M. Berz an K. Makino. COSY INFINITY version 8. Nuclear Instruments an Methos in Physics Research A, 7:338 33, 999. [] J.T. Betts. Survey of numerical methos for trajectory optimization. Journal of Guiance, Control, an Dynamics, :93 7, 998. [5] E. e Weert, Q.P. Chu, an J.A. Muler. Neural network output optimization using interval analysis. IEEE Transactions on Neural Networks, ():38 53, 9. [] Y. Fang. Optimal bicentere form. Reliable Computing, 9:9 3,. [7] Elon R. Hansen. On solving systems of equations using interval arithmetic. Mathematics of Computation, ():37 38, 98. [8] L. Jaulin, M. Kieffer, O. Dirit, an E. Walter. Applie Interval Analysis. Springer-Verlag Lonon Berlin Heielberg,. ISBN [9] Y. Lin an M.A. Statherr. Valiate solution of ODEs with parametric uncertainties. Appl. Numer. Math., 58:5, 7. [] K. Makino an M. Berz. New applications of Taylor moel methos. Automatic Differentiation of Algorithms: From Simulation To Optimization, pages 359 3,. [] K. Makino an M. Berz. Higher orer verifie inclusions of multiimensional systems by Taylor Moels. Nonlinear Analysis, 7:353 35,.

19 Reliable Computing, 3 [] K. Makino an M. Berz. Taylor moels an other valiate functional inclusion functions. International Journal of Pure an Applie Mathematics, ():379 5, 3. [3] K. Makino an M. Berz. Higher orer multivariate automatic ifferentiation an valiate computation of remainer bouns. Transactions on Computers, : 8, November 5. [] W.R. Mekwi. Iterative Methos for Roots of Polynomials. PhD thesis, Eeter College, University of Ofor,. [5] N.S. Neialkov. Interval Tools for ODEs an DAEs. In th GAMM - IMACS International Symposium on Scientific Computing, Computer Arithmetic an Valiate Numerics,. IEEE Conference Publications,. [] M. Neher, K.R. Jackson, an N.S. Neialkov. On Taylor moel base integration of ODEs. SIAM Journal on Numerical Analysis, 5:3, 7. [7] A. Neumaier. Taylor forms - use an limits. Reliable Computing, 9:3 79,. [8] Hermann Schichl an Arnol Neumaier. Interval analysis on irecte acyclic graphs for global optimization. Journal of Global Optimization, 33():5 5, 5.

20 3 e Weert et al, Polynomial Inclusion Functions # omains of PIF # omains 5 5 (V(VTS) V(PIF ))$ in percentage of $V(VTS).5 8 w[] (V(PIF ) V(PIF )) in percentage of $V(PIF ) relative V[%] 8.5 w[] 5 3 relative V[%] 8.5 w[] 3 Figure 8: Test case (f () = cos(π)) - Performance evaluation of the PIF compare to VTS. P IF is create using Proposition 3. only, while P IF is create using Proposition 3. too. V enotes the area between the bouns provie by the inclusion function.

21 Reliable Computing, 33 # omains of PIF (V(VTS) V(PIF ))$ in percentage of $V(VTS) # omains.5 8 w[] (V(PIF ) V(PIF )) in percentage of $V(PIF ) relative V[%] w[] relative V[%] w[] Figure 9: Test case (f () = ep( π )) - Performance evaluation of the PIF compare to VTS. P IF is create using Proposition 3. only, while P IF is create using Proposition 3. too. V enotes the area between the bouns provie by the inclusion function.

22 3 e Weert et al, Polynomial Inclusion Functions # omains of PIF 8 7 # omains 8 5 (V(VTS) V(PIF ))$ in percentage of $V(VTS).5 8 w[] (V(PIF ) V(PIF )) in percentage of $V(PIF ) relative V[%] w[] Figure : Test case 3 (f 3 () = n i= relative V[%] 8 8 sin(π i! + π i i! ), n = 5) - Performance evaluation of the PIF compare to VTS. P IF is create using Proposition 3. only, while P IF is create using Proposition 3. too. V enotes the area between the bouns provie by the inclusion function..5 w[] 5 3

23 Reliable Computing, 35 relative V[%] 8 (V(VTS) V(PIF))$ in percentage of $V(VTS) orer w[] Figure : Test case 3 (f 3 () = n i= t[s] 8 Computation time orer w[] VTS PIF sin(π i! + π i i! ), n = 5) - Performance evaluation of the PIF (Algorithm ) compare to VTS. V enotes the area between the bouns provie by the inclusion function.

24 3 e Weert et al, Polynomial Inclusion Functions (V(VTS) V(PIF))$ in percentage of $V(VTS) Figure : Test case 3 (f 3 () = relative V[%] orer n i= w[] 8 sin(π i! + π i i! ), n = 5) - Performance evaluation of the PIF (Algorithm ) compare to two VTSs. One VTS is use to form an inclusion of omain [inf[], mi[]], while the other is use for omain [mi[], sup[]]. V enotes the area between the bouns provie by the inclusion function. For the bottom plot, values lower than -% have been set to -%.

25 Reliable Computing, 37 (V(VTS) V(PIF))$ in percentage of $V(VTS) Figure 3: Test case 3 (f 3 () = relative V[%] orer n i= w[] 8 sin(π i! + π i i! ), n = ) - Performance evaluation of the PIF (Algorithm ) compare to two VTSs. One VTS is use to form an inclusion of omain [inf[], mi[]], while the other is use for omain [mi[], sup[]]. V enotes the area between the bouns provie by the inclusion function. For the bottom plot, values lower than -% have been set to -%.