Optimization of the Determinant of the Vandermonde Matrix and Related Matrices

Transcription

1 Methodol Comput Appl Proa 018) 0: Optimization of the Determinant of the Vandermonde Matrix and Related Matrices Karl Lundengård 1 Jonas Östererg 1 Sergei Silvestrov 1 Received: 30 Octoer 015 / Revised: 1 Septemer 017 / Accepted: 8 Septemer 017 / Pulished online: 14 Novemer 017 The Authors) 017 This article is an open access pulication Astract The value of the Vandermonde determinant is optimized over various surfaces, including the sphere, ellipsoid and torus Lagrange multipliers are used to find a system of polynomial equations which give the local extreme points in its solutions Using Gröner asis and other techniques the extreme points are given either explicitly or as roots of polynomials in one variale The ehavior of the Vandermonde determinant is also presented visually in some interesting cases Keywords Vandermonde determinant Optimization Gröner asis Orthogonal polynomials Ellipsoid Optimal experiment design Homogeneous polynomials Mathematics Suject Classification 010) 33C45 11C0 15B99 08B99 1 Introduction In this paper we will consider the extreme points of the Vandermonde determinant on various surfaces The examination is primarily motivated y mathematical curiosity ut the techniques used here are likely to e extensile to some prolems related to optimal experiment design for polynomial regression, see Section 3 Karl Lundengård karllundengard@mdhse Jonas Östererg jonasostererg@mdhse Sergei Silvestrov sergeisilvestrov@mdhse 1 Division of Applied Mathematics, UKK, Mälardalen University, Högskoleplan 1, Box 883, 71 3 Västerås, Sweden

2 1418 Methodol Comput Appl Proa 018) 0: To the authors knowledge this prolem has previously een examined on cues see Section 3) and spheres, see Szegő 1939) Here we will consider some techniques to extend some of these results to other surfaces such as ellipsoids, cylinders, p-norm spheres and other surfaces defined y homogeneous polynomials Our examination will mostly e restricted to three dimensions ut many of the techniques can in principle e extended to higher dimensions The Vandermonde Matrix A rectangular Vandermonde matrix of size m n is determined y n values x = x 1,,x n ) and is defined y [ ] V mn x) = xj i 1 = x 1 x x n mn 1) x m 1 1 x m 1 xn m 1 Note that some authors use the transpose of this as the definition and possily also let indices run from 0 All entries in the first row of Vandermonde matrices are ones and y considering 0 0 = 1 this is true even when some x j is zero In this paper we will primarily consider square Vandermonde matrices and for convenience we will use the notation V n x) = V nn x) The determinant of the Vandermonde matrix is well known Theorem 1 The determinant of square Vandermonde matrices has the form det V n x) v n x) = x j x i ) ) 1 i<j n This determinant is also simply referred to as the Vandermonde determinant or Vandermonde polynomial or Vandermondian Vein and Dale 1999) In this paper we will use the method of Lagrange multipliers to optimize the Vandermonde determinant over a surface For this purpose the following properties will e useful Lemma 1 The Vandermonde determinant is a homogeneous polynomial of degree nn 1) Proof Considering ) the numers of factor of v n x) is n i 1 = nn 1) Thus v n cx) = c nn 1) v n x) 3) 3 Application to D-optimal Experiment Designs for Polynomial Regression with a Cost-function Suppose an experiment is conducted where m data points from some compact interval, X R, i = 1,,,m, are used to create a polynomial regression model of degree

3 Methodol Comput Appl Proa 018) 0: n 1 A vector containing the data points, x m = x 1,x,,x m ) X m, is called a design and a design is said to e D-optimal if detm n x m )) detm n y m )) for all y X m where m m M m x) = m x i x n 1 i m x i m x i m m m m xi n x n 1 i x n i x n i is the Fischer information matrix, see Kiefer 1959) and Gaffke and Krafft 198) Optimal experiment design is often used to find the minimum numer of points needed for a certain model If we let m = n we get a interpolation prolem defined y a square Vandermonde matrix and the Fischer information matrix is M n x) = V n x) V n x) and since V n x) is an n n matrix detm n x)) = detv n x) ) detv n x)) = detv n x)) Thus the maximization of the determinant of the Fischer information matrix is equivalent to finding the extreme points of the determinant of a square Vandermonde matrix in some volume given y the set of possile designs, see Gaffke and Krafft 198) Optimal designs for various kinds of polynomial regression models are known, see Dette and Trampisch 010) for an overview The typical set of possile design is given y constraining each parameter of the model to e in a certain interval Usually these intervals are also normalized such that the vector of parameters x can e found in the n-dimensional cue x [ 1, 1] n When consider certain other sets of possile designs the results presented in this paper might e useful Suppose there is a cost-function associated with the data such that the total cost of the experiment eing elow some threshold value, gx) 1, defines some compact set, G ={x R m gx) 1}, such that G X m Since the Vandermonde determinant is a homogeneous polynomial for any c>1 v n x) > v n cx) the extreme points will e on the surface of the compact set and thus it is enough to consider the set of points defined y gx) = 1 4 Optimization using Gröner Bases Gröner ases together with algorithms to find them, and algorithms for solving a polynomial equation is an important tool that arises in many applications One such application is the optimization of polynomials over affine varieties through the method of Lagrange multipliers We will here give some main points and informal discussion on these methods as an introduction and to fix some notation Definition 1 Cox et al 1997)Letf 1,,f m e polynomials in R[x 1,,x n ]Theaffine variety Vf 1,,f m ) defined y f 1,,f m is the set of all points x 1,,x n ) R n such that f i x 1,,x n ) = 0forall1 i m

4 140 Methodol Comput Appl Proa 018) 0: When n = 3 we will sometimes use the variales x,y,z instead of x 1,x,x 3 Affine varieties are the common zeros of a set of multivariate polynomials Such sets of polynomials will generate a greater set of polynomials Cox et al 1997)y { m } f 1,,f m h i f i : h 1,,h m R[x 1,,x n ], and this larger set will define the same variety But it will also define an ideal a set of polynomials that contains the zero-polynomial and is closed under addition, and asors multiplication y any other polynomial) y If 1,,f m ) = f 1,,f m AGröner asis for this ideal is then a finite set of polynomials {g 1,,g k } such that the ideal generated y the leading terms of the polynomials g 1,,g k is the same ideal as that generated y all the leading terms of polynomials in I = f 1,,f m In this paper we consider the optimization of the Vandermonde determinant v n x) over surfaces defined y a polynomial equation on the form s n x 1,,x n ; p; a 1,,a n ) n a i x i p = 1, 4) where we will select the constants a i and p to get ellipsoids in three dimensions, cylinders in three dimensions, and spheres under the p-norm in n dimensions The case of the ellipsoid is suitale for solution y Gröner asis methods, ut due to the existing symmetries the spheres are more suitale for other methods, as provided in Section 8 From 3) and the convexity of the interior of the sets defined y 4), under a suitale choice of the constant p and non-negative a i, it is easy to see that the optimal value of v n on n a i x i p 1 will e attained on n a i x i p = 1 And so, y the method of Lagrange multipliers we have that the minimal/maximal values of v n x 1,,x n ) on s n x 1,,x n ) 1 will e attained at points such that v n / x i λ s n / x i = 0for1 i n and some constant λ and s n x 1,,x n ) 1 = 0, Tyrrell Rockafellar 1993) For p = the resulting set of equations will form a set of polynomials in λ, x 1,,x n These polynomials will define an ideal over R[λ, x 1,,x n ], and y finding a Grner asis for this ideal we can use the especially nice properties of Grner ases to find analytical solutions to these prolems, that is, to find roots for the polynomials in the computed asis 5 Extreme Points on the Ellipsoid in Three Dimensions In this section we will find the extreme points of the Vandermonde determinant on the three dimensional ellipsoid given y ax + y + cz = 1, x,y,z) R 3 5) where a>0, >0, c>0 Using the method of Lagrange multipliers together with 5) and some rewriting gives that all stationary points of the Vandermonde determinant lie in the variety V = V ax + y + cz 1,ax+ y + cz, axz x)y x) yz y)y x) + czz y)z x))

5 Methodol Comput Appl Proa 018) 0: Computing a Gröner asis for V using the lexicographic order x > y > z give the following three asis polynomials: g 1 z) = a + )a ) ) 4a + ) a + c) + c) + 3c a + a + ) + 3ca ) z +3ca + + c) 4a + )a + c) + c) + a + )c + a + )c ) z 4 g y, z) = g 3 x, z) = c + c)a + c)a + + c) z 6, 6) ) a + ) a + c) + c) + ca + )a + + c) + c a + ) z +q 1 z 5 q z 3 a )a + )a + + 3c)y, 7) ) a + ) a + c) + c) + ca + )a + + c) + ac a + ) z q 1 z 5 + q z 3 aa )a + )a + + 3c)x, 8) q 1 = 9 c + c)a + c)a + + c), q = 3ca + + c)3a + 4a c + 3a + 6ac + 4ac + 4 c + 4c ) The calculation of this asis was done using software for symolic compupation Maple ) Since g 1 only depends on z and g and g 3 are first degree polynomial in y and x respectively the stationary points can e found y finding the roots of g 1 and then calculate the corresponding x and y coordinates A general formula can e found in this case since g 1 only contains even powers of z it can e treated as a third degree polynomial) ut it is quite cumersome and we will therefore not give it explicitly Note that in general the polynomials in the Gröner asis are not guaranteed to have real-valued roots For the ellipsoid case examined here it can e shown that the computed Gröner asis will only give real-valued coordinates Since the extreme points of the Vandermonde determinant is a suset of the points with coordinates given y the roots of the polynomials in the Gröner asis this means that we will not need to check that the found points are real-valued or not, we only need to check whether they are proper extreme points or not Lemma The polynomial equation system defined y 6) 8) will only have solutions with real-valued x, y and z Proof The discriminant is a useful tool for determining how many real roots low-level polynomials have Following Irving 004) the discriminant, p), of a third degree polynomial px) = c 0 + c 1 x + c x + c 3 x 3 is p) = 18c 1 c c 3 c 4 4c 3 c 4 + c c 3 4c 1c3 3 7c 1 c 4 and if p) is non-negative then all roots will e real ut not necessarily distinct) Since the first asis polynomial g 1 only contains terms with even exponents and is of degree 6 the polynomial g 1 defined y g 1 z ) = g 1 z) will e a polynomial of degree 3 whose roots are the square fo the roots of g 1 Calculating the discriminant of g 1 gives g 1 ) = 9a ) a + + 3c) a + + c) 4 ac 3 3a 3 + a 3 c + a 3 + a c c + c 3 ) + 61aca + + c) )

6 14 Methodol Comput Appl Proa 018) 0: Since a, and c are all positive numers it is clear that g 1 ) is non-negative Furthermore, since a, and c are positive numers all terms in g 1 with odd powers have negative coefficients and all terms with even powers have positive coefficients Thus if w<0 then g 1 w) > 0 and thus all roots must e positive Since g is a first order polynomial with respect to y a real-valued z will give a real-valued y Sinceg 3 is a first order polynomial with respect to x a real-valued z will give a real-valued x An illustration of an ellipsoid and the extreme points of the Vandermonde determinant on its surface is shown in Fig 1 6 Extreme Points on the Cylinder in Three Dimensions In this section we will examine the local extreme points on an infinitely long in 3 dimensions cylinder aligned with the x-axis In this case we do not need to use Gröner asis techniques since the prolem can e reduced to a one dimensional polynomial equation Consider the cylinder defined y y + cz = 1, where >0, c>0 9) Using the method of Lagrange multipliers gives the equation system v 3 x = 0, v 3 y = λy, v 3 z = λcz Taking the sum of each expression gives Comining 9) and10)gives c ) + 1 cz = 1 z =± c y + cz = 0 y = c z 10) 1 c y = + c 1 + c Fig 1 Illustration of the ellipsoid defined y x 9 + y 4 +z = 0 with the extreme points of the Vandermonde determinant marked Displayed in Cartesian coordinates on the right and in ellipsoidal coordinates on the left

7 Methodol Comput Appl Proa 018) 0: Thus the plane defined y 10) intersects with the cylinder along the lines { ) } c 1 1 l 1 = x,, x R = {x, r, s) x R}, + c c + c { ) } c 1 1 l = x,, x R = {x, r, s) x R} + c c + c Finding the stationary points for v 3 along l 1 : v 3 x,r, s) = x + v 3 x x,r, s) = x + From this it follows that Thus 1 + c c c ) ) x + 1 r + s), + c )) 1 c + c c r + s) v 3 x x,r, s) = 0 x = 1 + c 1 1 x 1 = + c c c ) c, c, ) c ) c is the only stationary point on l 1 An analogous argument shows that x = x 1 is the only stationary point on l An example of where these points are placed on the cylinder is shown in Fig 11) 7 Optimizing the Vandermonde Determinant on a Surface Defined y a Homogeneous Polynomial When using Lagrange multipliers it can e desirale to not have to consider the λ-parameter the scaling etween the gradient and direction given y the constraint) we demonstrate a simple way to remove this parameter when the surface is defined y an homogeneous polynomial Fig Illustration of the cylinder defined y y z = 1 with the extreme points of the Vandermonde determinant marked Displayed in Cartesian coordinates on the right and in cylindrical coordinates on the left

8 144 Methodol Comput Appl Proa 018) 0: Lemma 3 Let g : R n R n e a homogeneous polynomial such that gcx) = c k gx) with k = nn 1) such that gx) = 1, x R n defines a continuous ounded surface Let z R n e a point on the surface Then z can e written as z = cy where v n x i = g x=y x, i 1,,,n 1) i x=y and c = gy) 1 k, if and only if z a stationary point for the Vandermonde determinant Proof By the method of Lagrange multipliers the point y {x R n gx) = 1} is a stationary point for the Vandermonde determinant and only if v n x = λ g k x=y x, k 1,,,n k x=y for some λ R The stationary points on the surface given y gcx) = c k will e given y c nn 1) v n x = c k λ g k x=y x, k 1,,,n k x=y and if c is chosen such that λ = c n1 n) c k then the stationary points are defined y v n x k = g x k, k 1,,,n Suppose that y {x R n gx) = c k } is a stationary point for v n then the point given y z = cy where c = gy) k 1 will e a stationary point for the Vandermonde determinant and will lie on the surface defined y gx) = 1 Lemma 4 If z is a stationary point for the Vandermonde determinant on the surface gx) = 1 where gx) is a homogeneous polynomial then z is either a stationary point or does not lie on the surface Proof Since g x) = 1) k gx) is either 1 or 1 then v n x) = v n x) for any point, including z and the points in a neighourhood around it which means that if g x) = gx) then the stationary points are preserved and otherwise the point will lie on the surface defined y gx) = 1 instead of gx) = 1 A well-known example of homogeneous polynomials are quadratic forms If we let gx) = x Sx then gx) is a quadratic form which in turn is a homogeneous polynomial with k = If S is a positive definite matrix then gx) = 1 defines an ellipsoid Here will will demonstrate the use of Lemma 3 to find the extreme points on a rotated ellipsoid Consider the ellipsoid defined y 1 9 x y yz z = 1 13)

9 Methodol Comput Appl Proa 018) 0: then y Lemma 3 we can instead consider the points in the variety V = V xy + xz + y z 9 x, Finding the Gröner asis of V gives x + xy yz + z 5 4 y 3 4 z, xz y + yz + x 3 4 y 5 4 z ) g 1 z) = z6z + 1)6064z 7436z + 697), g y, z) = z z z y, g 3 x, z) = z z z x This system is not difficult to solve and the resulting points are: p 0 = 0, 0, 0), p 1 = 0, 1 6, 1 6 p = p 3 = 45 ), 361, , ), 7 361, , ) 7 45 The point p 0 is not a valid solution since it does and does not lie on any ellipsoid defined y 5) and can therefore e discarded By Lemma 4 there are also three more stationary points p 4 = p 1, p 5 = p and p 6 = p 3 Rescaling each of these points according to Lemma 3 gives q i = gp i ) which are all points on the ellipsoid defined y gx) = 1 The result is illustrated in Fig 3 Note that this example gives a simple case with a small Gröner asis that is small and easy to find Using this technique for other polynomials and in higher dimensions can require significant computational resources Fig 3 Illustration of the ellipsoid defined y 13) with the extreme points of the Vandermonde determinant marked Displayed in Cartesian coordinates on the right and in ellipsoidal coordinates on the left

10 146 Methodol Comput Appl Proa 018) 0: The Vandermonde Determinant on p-norm Spheres The optimization of the Vandermonde determinant on the sphere p = ) and on the cue p = ) lends themselves to methods in orthogonal polynomials In fact, as shown y Stieltjes and recaptured y Szegő 1939), and presented in more detail in and extended in Lundengård et al 013) there is a fairly straightforward solution derived from electrostatic considerations, which implicitly deals with the Vandermonde determinant Consider the optimization of v n x) over the sphere n s n x) = x p i = 1, for suitale choices for p even) Instead of optimizing v n we are free to optimize ln v n over the sphere to the same effect v n x) = 0 is not a solution, all x i are pair-wise distinct) This leaves us with the set of equations y Lagrange multipliers ln v n = λ s n, s n = 1, 14) x k x k where the left-most equation holds for 1 k n It is easy to show that the partial derivatives can e written ln v n x k = n i =k 1 x k x i 15) From this it is easy to show that 15) can e rewritten y introducing the univariate polynomial P n x) = n x x i ) as ln v n = 1 P x k ) x k P x k ) Now the leftmost equation of 14) can e written or more succinct 1 P x k ) P x k ) = λ s n, x k P x k ) λ s n x k P x k ) = 0, 16) In the case p = weareluckysince16) ecomes, y introducing the new multiplier ρ P x) + ρ n xp x) ) x=xk = 0, 17) and since the left part of this equation is a polynomial of degree n and has roots x 1,,x n we must have P x) + ρ n xp x) + σ n Px) = 0, 18) for some ρ n,σ n that may depend on n Now if choose Px)to e monic, note that ρ n = 0, and require us to e on the sphere we get Px) = x n 1 xn +, and y identifying coefficients we get ρ n and σ n : P x) + n1 n)xp x) + n n 1)P x) = 0, which is a nice and well known form of differential equation and defines a sequence of orthogonal polynomials that are rescaled Hermite polynomials Szegő 1939), so we can find a recurrence relation for P n+1 in terms of P n and P n 1, and we can, for a fixed n, construct the coefficients of P n recursively, without explicitly finding P 1,,P n 1

11 Methodol Comput Appl Proa 018) 0: Now, this is for p = For p = 4 we continue from 16) instead with ) P x) + ρ n x 3 P x) x=xk = 0 19) Now the polynomial in x in the left part of this equation has shared roots with Px)and so y the same method as for p = weget: P x) + ρ n x 3 P x) + σ n x + τ n x + υ n )P x) = 0 0) It is easy to show for the sphere under any p-norm that the extreme points of ln v n x) where x 1 < <x n are unique, see Szegő 1939) and Lundengård et al 013), this coupled with the symmetry relation ln v n x) =ln v n x), provides us with the property that the extreme points are symmetric in the sense that for all 1 i n we have that there exists a 1 j n such that x i = x j, for odd n we then have that x i = 0forsomei We thus get polynomials Px)on the form: Px) = x n + c n x n + c n 4 x n 4 +, with every other coefficient zero, for even n we have only even powers, for odd n we have odd powers By identifying powers in 0) we get that τ n xpx) will not share any powers with any other part of the equation and so τ n = 0 We can also y identifying coefficients get nρ n + σ n = 0 We now have P x) + ρ n x 3 P x) + nρ n x + υ n )P x) = 0 1) For n = wegetthespecificsystem + ρx 3 x) + ρx + υ)x + c 0 ) = 0, ut we actually don t need to calculate much here since it is easy do adapt the roots of x + c 0 to the sphere with p = 4, we get: P 4 x) = x 1 The case n = 3 is also easy and y symmetry we get a zero coordinate: P3 4 x) = x3 1 x The case n = 4 ecomes a it more interesting: 1x + c ) + ρx 3 4x 3 + c x) + 4ρx + υ)x 4 + c x + c 0 ) = 0, υ ρc )x υc 4ρc 0 )x + c + υc 0 ) = 0, This provides three equations Now letting t = x so that Pt) = t + c t + c 0 = t t 1 )t t ) = t t 1 + t )t + t 1 t, gives us the last equation xi 4 = ti = c c 0) = 1 Solving this gives us P4 4 x) = x4 x Conclusion In this paper we have examined the extreme points of the Vandermonde determinant on various surfaces in three dimensions Explicit expressions for the placement of the extreme

12 148 Methodol Comput Appl Proa 018) 0: points on an ellipsoid aligned with the axis and a cylinder aligned with the x-axis were found using Gröner ases in Sections 5 and 6 A convenient way of rewriting the system of polynomial equations that the method of Lagrange multipliers gives was shown in Section 7 and it was illustrated how this rewrite could e used to find the extreme points on an ellipsoid not aligned with the coordinate system In Section 8 a method for finding the extreme points on spheres with p-norm was discussed, specifically the case when p = 4 for two, three and four dimensions Further work will e to extend the methods descried here to higher dimensions and also examine matrices related to the Vandermonde matrices, for instance the matrices descried in Section 3 Open Access This article is distriuted under the terms of the Creative Commons Attriution 40 International License which permits unrestricted use, distriution, and reproduction in any medium, provided you give appropriate credit to the original authors) and the source, provide a link to the Creative Commons license, and indicate if changes were made References Cox D, Little J, O Shea D 1997) Ideals, varieties, and algorithms Springer, New York Dette H, Trampisch M 010) A general approach to D-optimal designs for weighted univariate polynomial regression models Journal of the Korean Statistical Society 39:1 6 Gaffke N, Krafft O 198) Exact D-Optimum Designs for Quadratic Regression J R Stat Soc Ser B Methodol 443): Irving RS 004) Integers, Polynomials and rings Undergraduate texts in mathematics Springer, New York Kiefer JC 1959) Optimum experimental designs J R Stat Soc Ser B Methodol 1):7 319 Lundengård K, Östererg J, Silvestrov S 013) Extreme points of the Vandermonde determinant on the sphere and some limits involving the generalized Vandermonde determinant arxiv: Tyrrell Rockafellar R 1993) Lagrange multipliers and optimality SIAM Rev 35): Szegő G 1939) Orthogonal polynomials American Mathematics Society Vein R, Dale P 1999) Determinants and their applications in mathematical physics applied mathematical sciences, vol 134 Springer, New York Maple ) Maplesoft, a division of Waterloo Maple Inc, Waterloo, Ontario