The Bernstein polynomial basis: a centennial retrospective

Transcription

1 The Bernstein polynomial basis: a centennial retrospective a sociological study in the evolution of mathematical ideas Rida T. Farouki Department of Mechanical & Aerospace Engineering, University of California, Davis

2 synopsis 1912: Sergei Natanovich Bernstein constructive proof of Weierstrass approximation theorem 1960s: Paul de Faget de Casteljau, Pierre Étienne Bézier and the origins of computer-aided geometric design elucidation of Bernstein basis properties and algorithms 1980s: intrinsic numerical stability of the Bernstein form algorithms & representations for computer-aided design diversification of applications in scientific computing

3 Weierstrass approximation theorem Given any continuous function f(x) on an interval [ a, b ] and a tolerance ɛ > 0, a polynomial p n (x) of sufficiently high degree n exists, such that f(x) p n (x) ɛ for x [ a, b ]. Polynomials can uniformly approximate any continuous f(x), x [ a, b ]. Original (1885) proof by Weierstrass is existential in nature begins by expressing f(x) as a convolution 1 + [ f(x) = lim f(t) exp k 0 πk (t x)2 k 2 ] dt with a Dirac delta function, and relies heavily on analytic limit arguments.

4 Sergei Natanovich Bernstein ( ) (photo: Russian Academy of Sciences)

5 academic career of S. N. Bernstein 1904: Sorbonne PhD thesis, on analytic nature of PDE solutions (worked with Hilbert at Göttingen during academic year) 1913: Kharkov PhD thesis (polynomial approximation of functions) 1912: Comm. Math. Soc. Kharkov paper (2 pages): constructive proof of Weierstrass theorem introduction of Bernstein basis : Professor in Kharkov & Director of Mathematical Institute political purge: moved to USSR Academy of Sciences (Leningrad) : escapes to Kazakhstan during the siege of Leningrad : Steklov Math. Institute, Russian Academy of Sciences, Moscow edited complete works of Chebyshev (died 1968) collected works of Bernstein published in 4 volumes,

6 Bernstein s proof of Weierstrass theorem Russian school of approximation theory, founded by Chebyshev, favors constructive approximation methods over existential proofs given f(t) continuous on t [ 0, 1 ] define p n (t) = f(k/n) b n k(t), b n k(t) = k=0 ( ) n (1 t) n k t k k p n (t) = convex combination of sampled values f(0), f( 1 n ),..., f(1) f(t) p n (t) = O( 1 n ) for t [ 0, 1 ] = p n (t) converges uniformly to f(t) as n derivatives of p n (t) also converge to those of f(t) as n

7 S. N. Bernstein, Comm. Kharkov Math. Soc. (1912)

8 connection with probability theory basis function b n k (t) = ( n k ) (1 t) n k t k probability of k successes in n trials of random process with individual probability of success t in each trial binomial probability distribution non-negativity & partition-of-unity properties of b n k (t)

9 slow convergence of Bernstein approximations Bernstein polynomial approximations of degree n = 10, 30, 100, 300, 1000 to a triangular wave This fact seems to have precluded any numerical application of Bernstein polynomials from having been made. Perhaps they will find application when the properties of the approximant in the large are of more importance than the closeness of the approximation. Philip J. Davis, Interpolation and Approximation (1963)

10 Paul de Casteljau & Pierre Bézier (1960s) an emerging application computer-aided design Paul de Faget de Casteljau theory of courbes et surfaces à pôles developed at André Citroën, SA in the early 1960s de Casteljau s work unpublised (regarded as proprietary by Citroën) revealed to outside world by Wolfgang Böhm in mid-1980s Pierre Étienne Bézier implemented methods for computer-aided design and manufacturing at Renault during 1960s and 1970s Bézier published numerous articles and books describing his ideas basic problem: provide intuitive & interactive means for design and manipulation of free form curves and surfaces by computer, in the automotive, aerospace, and related industries identification of de Casteljau s and Bézier s ideas with Bernstein form of polynomials came later, through work of Forrest, Riesenfeld, et al.

11 de Casteljau Courbes et surfaces à pôles (Société Anonyme André Citroën, 1963) pôles = pilot points (interpolation of polynomials with polar forms)

12 de Casteljau barycentric coordinates de Casteljau s λ and µ = interval barycentric coordinates, with λ + µ = 1 example for t [ 0, 1 ] take λ = 1 t and µ = t, and expand (λ + µ) n 1 = [ (1 t) + t ] n = k=0 ( ) n (1 t) n k t k = k b n k(t) k=0 Bernstein basis {b n k (t)} is non negative and forms a partition of unity de Casteljau also considers extension to barycentric coordinates and multivariate polynomial bases on triangular and simplex domains

13 computer-aided design in the early 60s... the designers were astonished and scandalized. Was it some kind of joke? It was considered nonsense to represent a car body mathematically. It was enough to please the eye, the word accuracy had no meaning... reaction at Citroën to de Casteljau s ideas Citroën s first attempts at digital shape representation used a Burroughs E101 computer featuring 128 program steps, a 220-word memory, and a 5 kw power consumption! De Casteljau s insane persistence led to an increased adoption of computer-aided design methods in Citroën from 1963 onwards. My stay at Citroën was not only an adventure for me, but also an adventure for Citroën! P. de Casteljau

14 correspondence with de Casteljau (1991)

15 Bézier s point-vector form of a polynomial curve specify degree-n curve by initial point p 0 and n vectors a 1,..., a n r(t) = p 0 + k=1 a k f n k (t), f n k (t) = ( 1)k (k 1)! dk 1 (1 t) n 1 tk dt k 1 t a 2 a r(t) f 3 1(t) f 3 2(t) a f 3 3(t) 0.2 p Left: Bézier point-vector specification of a cubic curve. Right: cubic basis functions f 3 1 (t), f 3 2 (t), f 3 3 (t) associated with the vectors a 1 a 2, a 3. mischievous Bézier f n 1 (t),..., f n n (t) = basis of Onésime Durand!

16 control-point form of a Bézier curve n Forrest (1972) : fi n(t) = b n k(t), b n k(t) = k=i ( ) n (1 t) n k t k k p 1 p 2 p3 p 0 re-write as r(t) = p k b n k(t), k=0 p k = p k 1 + a k manipulate cure shape by moving control points p 0,..., p n

17 convex-hull, variation-diminishing, degree-elevation properties of the Bézier form r(t) = p k b n k(t), control points p 0,..., p n k=0

18 de Casteljau algorithm evaluates & subdivides r(t) initialize set t = τ and p 0 k = p k for k = 0,..., n for r = 1,..., n for j = r,..., n { p r j = (1 τ) pr 1 j 1 + τ pr 1 j } generates a triangular array of points { p r j } p 0 0 p 0 1 p 0 2 p 0 n p 1 1 p 1 2 p 1 n p 2 2 p 2 n p n n

19 p n n = evaluated point r(τ) on curve p 0 0, p 1 1,..., p n 1 n 1, pn n = control points for subsegment t [ 0, τ ] of r(t) p n n, p n 1 n,..., p 1 n, p 0 n = control points for subsegment t [ τ, 1 ] of r(t) p 5 p 0 p 4 p 1 r(τ) p 3 p 2

20 interlude... lost in translation warning sign on bathroom door in Beijing hotel

21 English on vacation in a Bucharest hotel lobby The elevator is being fixed for the next day. During that time we regret that you will be unbearable. in a Paris hotel elevator Please leave your values at the front desk. in a Zurich hotel Because of the impropriety of entertaining guests of the opposite sex in your bedroom, it is suggested that the lobby be used for this purpose. in an Acapulco restaurant The manager has personally passed all the water served here. in Germany s Schwarzwald It is strictly forbidden on our Black Forest camping site that people of different sex for instance, men and women live together in one tent unless they are married with each other for that purpose.

22 in an Athens hotel Guests are expected to complain at the office between 9 and 11 am daily. instructions for AC in Japanese hotel If you want just condition of warm in your room, please control yourself. in a Yugoslav hotel The flattening of underwear with pleasure is the job of the chambermaid. in a Japanese hotel You are invited to take advantage of the chambermaid. on the menu of a Swiss restaurant Our wines leave you nothing to hope for. in a Bangkok dry cleaners Drop your trousers here for best results. Japanese rental car instructions When passenger of foot heave in sight, tootle the horn. Trumpet him melodiously at first, but if he still obstacles your passage, then tootle him with vigor.

23 Bernstein basis functions b n k(t) t b n k (t) = ( n k ) (1 t) n k t k roots of multiplicity k and n k at t = 0 and t = 1

24 properties of the Bernstein basis b n k (t) = ( n k ) (1 t) n k t k, k = 0,..., n partition of unity : k=0 b n k(t) 1 non-negativity : b n k (t) 0 for t [ 0, 1 ] symmetry : b n k (t) = bn n k (1 t) recursion : b n+1 k (t) = t b n k 1 (t) + (1 t) bn k (t) unimodality : b n k (t) has maximum at t = k/n

25 properties of Bernstein form, p(t) = k=0 c k b n k(t) end-point values : p(0) = c 0 and p(1) = c n lower & upper bounds : min k c k p(t) max k c k variation diminishing : # roots = signvar(c 0,..., c n ) 2m derivatives & integrals : coefficients of p (t) & p(t) dt = differences & partial sums of c 0,..., c n recursive algorithms for subdivision, degree elevation, arithmetic operations, composition, resultants, etc. root isolation (subdivision & variation-diminishing property)

26 the plague of numerical instability... or, the temptation to kick the computer Do you ever want to kick the computer? Does it iterate endlessly on your newest algorithm that should have converged in three iterations? And does it finally come to a crashing halt with the insulting message that you divided by zero? These minor trauma are, in fact, the ways the computer manages to kick you and, unfortunately, you almost always deserve it! For it is a sad fact that most of us can more readily compute than think...

27 numerical stability of polynomials p(t) has coefficients c 0,..., c n in basis Φ = {φ 0 (t),..., φ n (t)} p(t) = c k φ k (t) k=0 how sensitive is a value or root of p(t) to perturbations of maximum relative magnitude ɛ in the coefficients c 0,..., c n? condition number for value of p(t) : δp(t) C Φ (p(t)) ɛ, C Φ (p(t)) = c k φ k (t) k=0 condition number for root τ of p(t) : δτ C Φ (τ) ɛ, C Φ (τ) = 1 p (τ) c k φ k (t) k=0

28 condition numbers for power and Bernstein forms c j = p(t) = a k t k = c k b n k(t) k=0 k=0 ( j j ( j ( k) n a k, t k) k = ( k) n b k) n j (t) k=0 j=k Theorem. C B (p(t)) C P (p(t)) for any polynomial p(t) and all t [ 0, 1 ]. Proof (triangle inequality). C B (p(t)) = c j b n j j (t) = j=0 j=0 k=0 ( j a k ( k) n b k) n j (t) = k=0 j=k k=0 ( j ( k) n a k b k) n j (t) a k t k = C P (p(t)).

29 Wilkinson s perfidious polynomial problem: compute the roots of the degree 20 polynomial p(t) = (t 1)(t 2) (t 20) = 20 k=0 a k t k using (software) floating-point arithmetic J. H. Wilkinson (1959), The evaluation of the zeros of ill-conditioned polynomials, Parts I & II, Numerische Mathematik 1, & The cosy relationship that mathematicians enjoyed with polynomials suffered a severe setback in the early fifties when electronic computers came into general use. Speaking for myself, I regard it as the most traumatic experience in my career as a numerical analyst. J. H. Wilkinson, The Perfidious Polynomial, in Studies in Numerical Analysis (1984)

30 root condition numbers for Wilkinson polynomial root power basis Bernstein basis

31 perturbed roots of Wilkinson polynomial ɛ = root power basis Bernstein basis ± i ± i ± i ± i ± i

32 evaluating Wilkinson s t = a 0 = a 1 t = a 2 t 2 = a 3 t 3 = a 4 t 4 = a 5 t 5 = a 6 t 6 = a 7 t 7 = a 8 t 8 = a 9 t 9 = a 10 t 10 = a 11 t 11 = a 12 t 12 = a 13 t 13 = a 14 t 14 = a 15 t 15 = a 16 t 16 = a 17 t 17 = a 18 t 18 = a 19 t 19 = a 20 t 20 = p(t) =

33 perturbation regions for p(t) = (t 1 6 ) (t 1) ε = ε = perturbed Bernstein form perturbed power form

34 optimal stability of Bernstein basis Ψ = {ψ 0 (t),..., ψ n (t)} and Φ = {φ 0 (t),..., φ n (t)} non negative on [ a, b ] Theorem. If ψ j (t) = M jk φ k (t) with M jk 0, k=0 then the condition numbers for the value of any degree n polynomial p(t) at any point t [ a, b ] in the bases Φ and Ψ satisfy C Φ (p(t)) C Ψ (p(t)). We say that the Φ basis is systematically more stable than the Ψ basis. Example: Φ = {b n 0(t),..., b n n(t)} and Ψ = {1, t,..., t n } in fact, the Bernstein basis is optimally stable (it is impossible to construct a basis on [ 0, 1 ] that is systematically more stable).

35 optimal stability (sketch) P n = set of all non negative bases for degree-n polynomials on [ a, b ]. For Φ, Ψ P n we write Φ Ψ if Ψ = M Φ for a non negative matrix M. The relation is a partial ordering of the set of non-negative bases P n. Theorem. Φ Ψ C Φ (p(t)) C Ψ (p(t)) for all p(t) P n and t [ a, b ]. Definition. Φ is a minimal basis in P n if no Ψ exists, such that Ψ Φ. A minimal basis in P n is optimally stable it is impossible to construct a non negative basis on [ a, b ] that is systematically more stable. Theorem. The Bernstein basis is minimal in P n, and is optimally stable. It is the only minimal basis whose basis functions have no roots in (a, b).

36 condition numbers can be very large!

37 least-squares polynomial approximation minimize 1 0 orthogonal basis [ f(t) p n (t) ] 2 dt, p n (t) = 1 0 φ j (t) φ k (t) dt = { = a k = 1 1 f(t) φ k (t) dt β k 0 β k a k φ k (t) k=0 j = k 0 j k permanence of coefficients: a 0,..., a n unchanged when n n + 1 orthogonality impossible for non negative bases, but Bernstein basis is intimately related to Legendre basis

38 Legendre and Bernstein bases on t [ 0, 1 ] recurrence relation L 0 (t) = 1, L 1 (t) = 2t 1 (k + 1)L k+1 (t) = (2k + 1)(2t 1) L k (t) k L k 1 (t) Rodrigues formula Bernstein form L k (t) = L k (t) = ( 1)k k! d k [ (1 t)t ]k dtk k ( ) k ( 1) k+i b k i (t) i i=0 Pascal s triangle with alternating signs!!

39 Bernstein form of the Legendre polynomials L 0 (t) = 1 b 0 0(t), L 1 (t) = 1 b 1 0(t) + 1 b 1 1(t), L 2 (t) = 1 b 2 0(t) 2 b 2 1(t) + 1 b 2 2(t), L 3 (t) = 1 b 3 0(t) + 3 b 3 1(t) 3 b 3 2(t) + 1 b 3 3(t), L 4 (t) = 1 b 4 0(t) 4 b 4 1(t) + 6 b 4 2(t) 4 b 4 3(t) + 1 b 4 4(t), L 5 (t) = 1 b 5 0(t) + 5 b 5 1(t) 10 b 5 2(t) + 10 b 5 3(t) 5 b 5 4(t) + 1 b 5 5(t), Bernstein form of Legendre polynomial derivatives e.g., L 4 (t) L 4 (t) = 1 b 4 0(t) 4 b 4 1(t) + 6 b 4 2(t) 4 b 4 3(t) + 1 b 4 4(t), 1 2 L 4(t) = 5 b 3 0(t) 10 b 3 1(t) + 10 b 3 2(t) 5 b 3 3(t), 1 3 L 4(t) = 15 b 2 0(t) 20 b 2 1(t) + 15 b 2 2(t), 1 3 L 4 (t) = 35 b 1 0(t) 35 b 1 1(t), 1 5 L 4 (t) = 70 b 0 0(t),

40 Legendre Bernstein basis transformations p(t) = c j = M jk = ( 1 ) n k a k L k (t) = k=0 M jk a k, a j = k=0 min(j,k) i=max(0,j+k n) M 1 jk = 2j + 1 n + j + 1 c k b n k(t) k=0 k=0 ( 1) k+i ( j i ( ) j n ( 1) j+i k i=0 M 1 jk c k )( k i )( ) n k j i ( )( ) j j i i ( ) n + j k + i condition number C p (M) = M p M 1 p, C 1 (M) = 2 n > C (M)

41 condition numbers for basis transformations matrix condition number Bernstein Legendre Bernstein power Bernstein Hermite Hermite power polynomial degree

42 extension to rational forms rational Bézier curve r(t) = w k p k b n k(t) k=0 w k b n k(t) k=0 defined by control points p 0,..., p n and scalar weights w 0,..., w n set of rational curves is closed under projective transformations conic segments as rational quadratic Bézier curves (w 0 = w 2 = 1) w 1 < 1 (ellipse) w 1 = 1 (parabola) w 1 > 1 (hyperbola)

43 bivariate & multivariate generalizations p p T 2 T 3 T 1 p p 1 p 2 + p barycentric coordinates: (u, v, w) = (area(t 1), area(t 2 ), area(t 3 )) area(t ) 1 = (u + v + w) n = b n ijk(u, v, w), b n ijk(u, v, w) = n! i!j!k! ui v j w k i+j+k=n triangular surface patch: r(u, v, w) = i+j+k=n p ijk b n ijk(u, v, w)

44 bivariate de Casteljau algorithm generates tetrahedral array evaluates and subdivides r(u, v, w) r = r = r = r =

45 generalization to B-spline basis continuous domain [ 0, 1 ] partitioned domain [ t 0, t 1,..., t N+n+1 ] B 3 0(t) B 3 1(t) B 3 2(t) B 3 3(t) B 3 N 2(t) B 3 N 1(t) B 3 N(t) t 0 =t 1 =t 2 =t 3 t 4 t 5 t N t N+1 =t N+2 =t N+3 =t N+4 retain partition-of-unity, non-negativity, recursion properties + compact support & control over continuity (multiple knots) local shape modification C 2 embedded linear segment

46 scientific computing applications real solutions of systems of algebraic equations; identifying extrema or bounds on constrained or unconstrained polynomial functions in one or several variables (optimization) using Bernstein basis properties robust stability of dynamic systems with uncertain physical parameters (Kharitonov generalization of Routh-Hurwitz criterion) definition of barycentric coordinates and partition-of-unity polynomial basis functions over general polygon or polytope domains for use in the finite-element and meshless analysis methods modelling of inter-molecular potential energy surfaces; design of filters for signal processing applications; inputs to neurofuzzy networks modelling non-linear dynamical systems; reconstruction of 3D models and calibration of optical range sensors

47 closure 100 years have elapsed since introduction of Bernstein basis Bernstein form was limited to theory, rather than practice, of polynomial approximation for 50 years after its introduction applications in design, rather than approximation, pioneered 50 years ago by de Casteljau and Bézier now universally adopted as a fundamental representation for computer-aided geometric design applications optimally stable basis for polynomials defined over finite domains Bernstein basis intimately related to Legendre orthogonal basis increasing adoption in diverse scientific computing applications In theory, there is no difference between theory and practice. In practice, there is.... Yogi Berra