Shape Restricted Smoothing Splines via Constrained Optimal Control and Nonsmooth Newton s Methods

Transcription

1 Shape Restricted Smoothing Splines via Constrained Optimal Control and Nonsmooth Newton s Methods Jinglai Shen and Teresa M. Lebair Original: January 24, 214; Revised: June 6, 214 Abstract Shape restricted smoothing splines receive considerable attention, motivated by a wide range of important applications in science and engineering. In this paper, we consider smoothing splines subject to general linear dynamics and control constraints, and formulate them as finitehorizon constrained linear optimal control problems with unknown initial state and control. By exploring techniques from functional and variational analysis, optimality conditions are developed in terms of variational inequalities. Due to the control constraints, the optimality conditions give rise to a nonsmooth B-differentiable equation of an optimal initial condition, whose unique solution completely determines the shape restricted smoothing spline. A modified nonsmooth Newton s algorithm with line search is used to solve this equation; detailed convergence analysis of the proposed algorithm is presented. In particular, using techniques from nonsmooth analysis, polyhedral theory, and switching systems, we show the global convergence of the algorithm when a shape restricted smoothing spline is subject to a general polyhedral control constraint. 1 Introduction Spline models are extensively studied in approximation theory, numerical analysis, and statistics with broad applications in science and engineering. Informally speaking, a univariate spline model gives a piecewise polynomial curve that best fits a given set of data. Such a spline can be attained via efficient numerical algorithms and enjoys favorable analytical and statistical properties [9]. A number of variations and extensions of this model have been developed, e.g., penalized polynomial splines [36] and smoothing splines [43]. Specifically, the smoothing spline model is a smooth function f : [, 1] R in a suitable function space that minimizes the following objective functional: 1 n n ) 1 2 fti ) y i + λ f m) t)) 2dt, 1) where y i are data points at t i [, 1], i = 1,..., n, f m) denotes the m-th derivative of f, and λ > is a penalty parameter that characterizes a tradeoff between data fidelity and the smoothness of f. We refer the reader to [43] and references therein for statistical properties of smoothing splines. From a control systems point of view, the smoothing spline model 1) is closely related to the finite-horizon linear quadratic optimal control problem by treating f m) as a control input [12]. This has led to a highly interesting spline model defined by a linear control system called a control Department of Mathematics and Statistics, University of Maryland Baltimore County, Baltimore, MD 2125, U.S.A. shenj@umbc.edu. This research is supported by NSF grants CMMI-1384 and DMS Department of Mathematics and Statistics, University of Maryland Baltimore County, Baltimore, MD 2125, U.S.A. ei44375@umbc.edu. 1

2 theoretic spline [12]. It is shown in [12] and the references therein, e.g., [16, 4, 46], that a number of smoothing, interpolation, and path planning problems can be incorporated into this paradigm and studied using control theory and optimization techniques on Hilbert spaces with efficient numerical schemes. Other relevant approaches include control theoretic wavelets [15]. Although many important results for unconstrained or equality constrained spline models are available, various biological, engineering and economic systems contain functions whose shape and/or dynamics are subject to inequality constraints, e.g., the monotone and convex constraints. Examples include monotone regulatory functions in genetic networks [37] and a shape restricted function in an attitude control system [3]. Other applications are found in reliability engineering e.g., survival/hazard functions), medicine e.g., dose-response curves), finance e.g., option/delivery price), and astronomy e.g., galaxy mass functions). Incorporating the knowledge of shape constraints into a construction procedure improves estimation efficiency and accuracy [29]. This has raised surging interest in the study of constrained splines and shape restricted estimation in statistics and other related fields; see some recent results [23, 36, 38, 39, 44]. The present paper focuses on shape restricted smoothing splines formulated as constrained linear optimal control problems with unknown initial state and control. Two types of constraints arise: i) control constraints; and ii) state constraints. While several effective numerical methods have been developed for state constrained optimal control problems in [14, 19, 45] also see [18] for a recent survey), we focus on control constraints in this paper, since a variety of shape constraints, which may be imposed on derivatives of a function, can be easily formulated as control constraints. It should be noted that a control constrained optimal control problem is inherently nonsmooth, and thus is considerably different from a classical unconstrained) linear optimal control problem such as LQR. Moreover, the goal of the shape constrained spline problem is to find an optimal initial condition and an open-loop like optimal control that best fit sample data instead of finding an optimal state feedback as in LQR. Furthermore, most of the current literature on control constrained smoothing splines focuses on relatively simpler linear dynamics and special control constraints, e.g., [1, 12, 16, 23, 41], and many critical questions remain open in analysis and computation when general dynamics and control constraints are taken into account. For example, a widely used approach in the literature concentrates on shape restricted smoothing splines whose linear dynamics are defined by certain nilpotent matrices, and whose control constraints are a cone in R [11, 22]. In this case, the smoothing spline is a piecewise continuous polynomial with a known degree. Hence the computation of the smoothing spline boils down to determining parameters of a polynomial on each interval, which can be further reduced to a quadratic or semidefinite program that attains efficient algorithms [1, 12]. However, this approach fails to handle general linear dynamics and control constraints, since the solution form of a general shape restricted smoothing spline is unknown a priori. Therefore new tools are needed to handle more general dynamics and general control constraint induced nonsmoothness. In this paper, we develop new analytical and numerical results for smoothing splines subject to general dynamics and control constraints by using optimal control and nonsmooth optimization techniques. The major contributions of the paper are summarized as follows: 1. Optimal control formulation and analysis. By using the Hilbert space method and variational techniques, optimality conditions are established for shape restricted smoothing splines in the form of variational inequalities. These optimality conditions yield a nonsmooth equation of an optimal initial condition; it is shown that the unique solution of this equation completely determines an optimal control and thus the desired smoothing spline cf. Theorem 3.2 and Corollary 3.1). 2. Numerical computation and convergence analysis. To solve the above mentioned equation, we show the B-differentiability and other nonsmooth properties of this equation. A modified nonsmooth Newton s algorithm with line search [24] is invoked to solve the equation. This algorithm does not 2

3 require knowing the solution form of a smoothing spline a priori. However, the convergence of the original nonsmooth Newton s method in [24] relies on several critical assumptions, including the boundedness of level sets and global existence of direction vectors for a related equation. The verification of these assumptions for shape restricted smoothing splines turns out to be rather nontrivial, due to dynamics and constraint induced complexities. By using various techniques from nonsmooth analysis, polyhedral theory, and piecewise affine switching systems, we establish the global convergence of the proposed algorithm for a general polyhedral control constraint under mild technical conditions cf. Theorems ). The paper is organized as follows. In Section 2, we formulate a shape restricted smoothing spline as a constrained optimal control problem with optimality conditions developed in Section 3. A nonsmooth Newton s algorithm for the smoothing spline is proposed in Section 4; its convergence analysis and numerical results are presented in Section 5 and Section 6 respectively, for polyhedral control constraints. Finally, conclusions are made in Section 7. Notation. We introduce the following notation used throughout the paper. Let, denote the inner product on the Euclidean space. Let I S denote the indicator function for a set S. Let denote the orthogonality of two vectors in R n, i.e., a b implies a T b =. For a closed convex set K in R n, Π K z) denotes the Euclidean projection of z R n onto K. It is known that Π K ) is Lipschitz continuous on R n with the Lipschitz constant L = 1 when the Euclidean norm is used [13]. Throughout the paper, let be the Lebesgue integral. For a matrix M, M j denotes its jth row and KerM) denotes the null space of M. Finally, for a function F : R n R n and a closed convex set K in R n, let VIK, F ) be the variational inequality problem whose solution is z K if z z, F z ) for all z K. We shall use SOLK, F ) to denote the solution set of VIK, F ). A VI is a highly nonlinear and nonsmooth problem, even if F is linear and K is polyhedral [13]. 2 Shape Restricted Smoothing Splines: Constrained Optimal Control Formulation Consider the linear control system on R l subject to control constraint: ẋ = Ax + Bu, y = Cx, 2) where A R l l, B R l m, and C R p l. Let Ω R m be a closed convex set. The control constraint is given by u L 2 [, 1]; R m ) and ut) Ω for almost all t [, 1], where L 2 [, 1]; R m ) is the space of square R m -valued Lebesgue) integrable functions. We denote this constrained linear control system by ΣA, B, C, Ω). Define the set of permissible controls, which is clearly convex: W : = { u L 2 [, 1]; R m ) ut) Ω, a.e. [, 1] Let the underlying function f : [, 1] R p be the output ft) := Cxt) for an absolutely continuous trajectory xt) of ΣA, B, C, Ω), which can be completely determined by its initial state and control. Consider the following generalized) regression problem on the interval [, 1]: y i = ft i ) + ε i, i =, 1,..., n, 3) where t i s are the pre-specified design points with = t < t 1 < < t n = 1, y i R p are samples, and ε i R p are noise or errors. Given the sample observation t i, y i ) n i=, and w i >, i = 1,..., n such that n w i = 1 e.g., w i = t i t i 1 ). Define the cost functional n J := w i y i Cxt i ) λ ut) 2 2dt, 4) 3 }.

4 where λ > is the penalty parameter. The goal of a shape restricted smoothing spline is to find an absolutely continuous trajectory xt) which is determined by its initial state and control) that minimizes the cost functional J subject to the dynamics of the linear control system ΣA, B, C, Ω) 2) and the control constraint u W. Remark 2.1. Let R R m m be a symmetric positive definite matrix. A more general cost functional n J := w i y i Cxt i ) λ u T t)rut)dt 5) may be considered. However, a suitable control transformation will yield an equivalent problem defined by the cost functional 4). In fact, let R = P T P for an invertible matrix P. Let vt) = P ut), Ω = P Ω, and W : = { v L 2 [, 1]; R m ) vt) Ω, a.e. [, 1] }. Clearly, Ω remains closed and convex, and W is still convex. Therefore the constrained optimal control problem defined by 5) for the linear control system ΣA, B, C, Ω) is equivalent to that defined by 4) with u replaced by v for the linear system ΣA, BP 1, C, Ω ) subject to the constraint v, x ) W R l. Example 2.1. The constrained linear control model 2) covers a wide range of estimation problems subject to shape and/or dynamical constraints. For instance, the standard monotone regression problem is a special case of the model 2) by letting the scalars A =, B = C = 1, and Ω = R +. Another case is the convex regression, for which [ ] [ ] [ ] 1 A = R 2 2, B = R 2, C T 1 = R 2, Ω = R Optimality Conditions of Shape Restricted Smoothing Splines This section develops optimality conditions for the finite-horizon constrained optimal control problem 4) using the Hilbert space techniques. We first introduce the following functions P i : [, 1] R p m inspired by [12]: P i t) := { Ce At i t) B, if t [, t i ], if t > t i, i = 1,..., n. Hence, ft i ) = Cxt i ) = Ce At i x + P i t)ut)dt, i = 1,..., n, where x denotes the initial state of xt). Define the set P := W R l. It is easy to verify that P is convex. The constrained optimal control problem is formulated as where J : P R + is given by Ju, x ) := n w i Ce At i x + inf Ju, x ), 6) u,x ) P 2 P i t)ut)dt y i + λ 2 ut) 2 2dt. For given design points {t i } n in [, 1], we introduce the following condition: Ce At 1 Ce At 2 H.1 : rank. = l. Ce Atn 4

5 It is easy to see, via t i [, 1] for all i, that if C, A) is an observable pair, then the condition H.1 holds for all sufficiently large n. Under this condition, the existence and uniqueness of an optimal solution can be shown via standard arguments in functional analysis, e.g., [2, 4, 6, 17, 21, 42]. We present its proof in the following theorem for self-containment. Theorem 3.1. Suppose {t i, y i )}, {w i }, and λ > are given. Under the condition H.1, the optimization problem 6) has a unique optimal solution u, x ) P. Proof. Consider the Hilbert space L 2 [, 1]; R m ) R l endowed with the inner product u, x), v, z) := 1 ut t)vt)dt+x T z for any u, x), v, z) L 2 [, 1]; R m ) R l. Its induced norm satisfies u, x) 2 := u 2 L 2 + x 2 2, where u 1 1/2 L 2 := t)ut)dt) ut for any u L2 [, 1]; R m ) and 2 is the Euclidean norm on R l. The following properties of J : L 2 [, 1]; R m ) R l R + can be easily verified via the positive definiteness of the matrix n w ice At i ) T Ce At i ) R l l due to H.1: i) J is coercive, i.e., for any sequence {u k, x k )} with u k, x k ) as k, Ju k, x k ) as k. ii) J is strictly convex, i.e., for any u, x), v, z) L 2 [, 1]; R m ) R l, Jαu, x) + 1 α)v, z)) < αju, x) + 1 α)jv, z), α, 1). Pick an arbitrary ũ, x) P and define the level set S := {u, x) P : Ju, x) Jũ, x)}. Due to the convexity and the coercive property of J, S is a convex and L 2 -norm) bounded set in L 2 [, 1]; R m ) R l. Since the Hilbert space L 2 [, 1]; R m ) R l is reflexive and self dual, it follows from Banach-Alaoglu Theorem [21] that an arbitrary sequence {u k, x k )} in S with u k W and x k R l has a subsequence {u k, x k )} that attains a weak*, thus weak, limit u, x ) L 2 [, 1]; R m ) R l. Clearly, x R l. Without loss of generality, we assume that for each u k, u k t) Ω for all t [, 1]. Therefore, Ce At i x k + P it)u k t)dt converges to CeAt i x + P it)u t)dt for each i. Next we show that u W via the closedness and convexity of Ω. In view of the weak convergence of u k ) to u, it follows from Mazur s Lemma [28, Lemma 1.19] that there exists a sequence of convex combinations of u k ), denoted by v k), that converges to u strongly in L 2 [, 1]; R m ), i.e., for each k, there exist an integer p k k and real numbers λ k,j, k j p k with p k j=k λ k,j = 1 such that v k = p k j=k λ k,ju k, and v k u L2 as k. Since each u k t) Ω, t [, 1], the same holds for each v k via the convexity of Ω. Furthermore, due to the strong convergence of v k ) to u i.e., in the L 2 -norm), v k ) converges to u in measure [3, pp. 69], and hence has a subsequence that converges to u pointwise almost everywhere on [, 1] cf. [3, Theorem 7.6] or [17, Theorem 5.2]). Since Ω is closed, u t) Ω for almost all t [, 1]. This shows that u W. Furthermore, by using the L 2 -norm) boundedness of u k ) and the triangle inequality for the L 2- norm, it is easy to show that for any η >, there exists K N such that u 2 L 2 u k 2 L 2 +η, k K. These results imply that for any ε >, Ju, x ) Ju k, x k ) + ε for all k sufficiently large. Consequently, Ju, x ) Jũ, x) such that u, x ) S. This thus shows that S is sequentially compact. In view of the strong) continuity of J, we see that a global optimal solution exists on S [21, Section 5.1, Theorem 2], and thus on P. Moreover, since J is strictly convex in u, x ) and the set P is convex, the optimal solution u, x ) must be unique. The next result provides the necessary and sufficient optimality conditions in terms of variational inequalities. In particular, the optimality conditions yield two equations: 7) and 8). It is shown in Corollary 3.1 that equation 7) implies that if x is known, then the smoothing spline can be determined inductively. Furthermore, the optimal initial state x can be solved from the nonsmooth) equation 8), for which a nonsmooth Newton s method will be used cf. Section 4). 5

6 Theorem 3.2. The pair u, x ) P is an optimal solution to 6) if and only if the following two conditions hold: where and u t) = Π Ω Gt, u t), x )/λ ), a.e. [, 1], 7) Lu, x ) =, 8) Gt, u t), x ) := Lu, x ) := n w i Pi T t) Ce At i x + n ) w i Ce At i T Ce At i x + P i t)u t)dt y i ), 9) P i t)u t)dt y i ). Proof. Let u, x ) P be arbitrary. Due to the convexity of P, u, x )+ε[u, x ) u, x )] P for all ε [, 1]. Further, since u, x ) is a global optimizer, we have Ju, x )+ε[u, x ) u, x )]) Ju, x ) for all ε [, 1]. Therefore Ju, x lim ) + ε[u, x ) u, x )]) Ju, x ) ε ε [ n = 2 w i Ce At i x + P i t)u t)dt y i, Ce At i x x ) + + λ ] ) u t) T u t) u t) dt. This thus yields the necessary optimality condition: for all u, x ) P, n w i Ce At i x + P i t)u t)dt y i, Ce At i x x ) + + λ ) P i t) u t) u t) dt ) P i t) u t) u t) dt ) u t) T u t) u t) dt. 1) This condition is also sufficient in light of the following inequality due to the convexity of J: Ju, x ) Ju, x Ju, x ) lim ) + ε[u, x ) u, x )]) Ju, x ), u, x ) P. ε ε We now show that the optimality condition 1) is equivalent to u t) u t), λu t) + Gt, u t), x ), a.e. [, 1], u W, 11) where Gt, u t), x ) is given in 9), and Lu, x ), x x, x R l. 12) Clearly, if 11) and 12) hold, then 1) holds. Conversely, by setting u = u, we have from 1) that n w i Ce At i x + P i t)u t)dt y i, Ce At i x x ), x R l. 6

7 Since x is arbitrary in R l, this yields 12) and thus 8). Furthermore, the condition 1) is reduced to u t) u t), λu t) + Gt, u t), x ) dt, u W. Let Gt, u t), x ) := λu t) + Gt, u t), x ). Since G L 2 [, 1]; R m ), u L 2 [, 1]; R m ), and Ω is closed and convex, it follows from [26, Section 2.1] that the above integral inequality is equivalent to the variational inequality 11), which is further equivalent to u t) SOL Ω, Gt,, x )), a.e. [, 1]. Hence, for almost all t [, 1], w u t), u t) + Gt, u t), x )/λ, w Ω. This shows u t) = Π Ω Gt, u t), x )/λ) a.e. [, 1]. Let ft, x ) denote the shape restricted smoothing spline for the given {y i}, i.e., ft, x ) := Ce At x + Ce At s) Bu s, x )ds, where u is the optimal control, and x is the optimal initial state. Corollary 3.1. The shape restricted smoothing spline ft, x ) satisfies n ) w i Ce At i T fti, x ) ) y i =, 13) and Gt, u t), x ) in 9) is given by, t [, t 1 ) Gt, u t), x ) = k w i Ce At i t) B ) T fti, x ) ) y i, t [tk, t k+1 ), k = 1,..., n 1 Proof. Note that ft i, x ) = CeAt i x + P is)u s, x )ds for i = 1,..., n. In light of 8) and the definition of Lu, x ), we obtain 13). To establish 14), we see from 13) that n = ) T ) w i Ce At i fti, x ) y i I [,ti ] 14), t [, t 1 ) k ) w i Ce At i T fti, x ) ) y i, t [tk, t k+1 ), k = 1,..., n 1. 15) Moreover, it follows from 9) and the definition of P i that Gt, u t), x ) = = = n ) T ) w i Ce Ati t) B I [,ti ] fti, x ) y i n T ) w i Ce At i e B) At I[,ti ] fti, x ) y i e At B By virtue of this and 15), we obtain 14). ) T n 7 ) T ) w i Ce At i fti, x ) y i I [,ti ].

8 This corollary shows that if the optimal initial condition x is known, then the smoothing spline ft, x ) can be determined inductively. In fact, on each interval [t k, t k+1 ), Gt, u t), x ), and thus u t), depends on ft i, x ) with t i t k only. This property will be exploited to compute the shape constrained smoothing splines in Section 4. We mention a few special cases of particular interest as follows. If K is a closed convex cone C, then z SOLK, F ) if and only if C z F z) C, where C is the dual cone of C. In particular, if K is the nonnegative orthant R n +, then z SOLK, F ) if and only if z F z), where the latter is called the complementarity problem cf. [8, 13] for details). Especially, when F z) is affine, i.e., F z) = Mz +q for a square matrix M and a vector q, then the complementarity problem becomes the linear complementarity problem LCP). Another special case of significant interest is when K is a polyhedron, namely, K = {z R n Dz b, Ez = d}, where D R r n, E R q n, and b R r, d R q. In this case, it is well known that z SOLK, F ) if and only if there exist multipliers χ R r, µ R q such that F z) D T χ + E T µ =, χ Dz b, Ez d = [13, Proposition 1.2.1]. Along with these results, we obtain the following optimality condition for u in terms of a complementarity problem when Ω is polyhedral. Proposition 3.1. Let Ω = {w R m Dw b} be a nonempty) polyhedron with D R r m and b R r. Then [ ] u t) = Gt, u t), x ) + D T χgt, u t), x )) /λ, a.e. [, 1], where D T χ : R m R m is a continuous piecewise affine function defined by the solution of the linear complementarity problem: χ λ 1 DD T χ λ 1 Dz b. Proof. It follows from u SOLΩ, Gt,, x )) a.e. [, 1], where Gt, u t), x ) = λu + Gu t), x ), and the above discussions that u is the optimal control if and only if for almost all t [, 1], there exists χ R r such that λu t) + Gu t), x ) D T χ =, and χ Du t) b. This is equivalent to the linear complementarity problem χ λ 1 DD T χ λ 1 z b, 16) where z := Gu t), x ). Due to the positive semidefinite plus PSD-plus) structure [35], it follows from complementarity theory [8] that for any z R m, the LCP 16) has a solution χz), and D T χz) is unique, which further implies that D T χ ) is continuous and piecewise affine [35]. 4 Computation of Shape Restricted Smoothing Splines via Nonsmooth Newton s Method In this section, we discuss the numerical issues of the shape restricted smoothing splines. As indicated below Corollary 3.1, in order to determine the smoothing spline ft, x ), it suffices to find the optimal initial state x, since once x is known, u and f can be computed recursively. In fact, it follows from Corollary 3.1 that ft, x ) is given by ft, x ) = Ce At x + t Ce At s) Bu s, x )ds, 17) 8

9 where u t, x ) = Π Ω ), t [, t 1 ) k Π Ω λ 1 w i Ce At i t) B ) T fti, x ) ) ) y i, t [t k, t k+1 ), k = 1,..., n 1 and ft, x ) satisfies H y,n x ) := n ) T ) w i Ce At i fti, x ) y i =. 18) To compute the optimal initial state x, we consider the equation H y,nz) =, where ft, z) in H y,n z) is defined by 17) with x replaced by z. The following lemma is a direct consequence of Theorem 3.1 and the definition of f. Lemma 4.1. For any given {t i, y i )}, {w i }, and λ > satisfying H.1, the equation H y,n z) = has a unique solution, which corresponds to the optimal initial state x of the smoothing spline ft, x ). It should be noted that the function H y,n : R l R l is nonsmooth in general, due to the constraint induced nonsmoothness of u t, x ) in x. However, the following proposition shows the Bouligand)-differentiability of ft, z) in z [13, Section 3.1]. Recall that a function G : R l R l is B-differentiable if it is Lipschitz continuous and directionally differentiable on R l, namely, for any z R l and any direction vector d R l, the following one-sided) directional derivative exists G Gz + τd) Gz) z; d) := lim. τ τ Proposition 4.1. Assume that Π Ω : R m R m is directionally differentiable on R m. For any given {t i, y i )}, {w i }, λ >, and z R l, ft, z) is B-differentiable in z for any fixed t [, 1]. Proof. We prove the B-differentiability of ft, z) in z by induction on the intervals [t k, t k+1 ], k =, 1,..., n 1. Consider t [, t 1 ] first. Since u t, z) = Π Ω ), t [, t 1 ) and ft, z) is continuous in t, ft, z) = Ce At z + t CeAt s) B Π Ω )ds, t [, t 1 ], which is clearly Lipschitz continuous and directionally differentiable, thus B-differentiable, in z for any fixed t [, t 1 ]. Now assume that ft, ) is B-differentiable for all t [, t 1 ] [t k 1, t k ], and consider the interval [t k, t k+1 ]. Note that for any t [t k, t k+1 ), the optimal control k T ) u t, z) = Π Ω λ 1 w i Ce B) ) Ati t) fti, z) y i. 19) Since the functions Π Ω ) and ft i, ), i = 1,..., k are all B-differentiable, it follows from [13, Proposition 3.1.6] that the composition given in u t, z) remains B-differentiable in z for each fixed t [t k, t k+1 ). For a given direction vector d R l and a given τ, u t, z + τd) is continuous in t on [t k, t k+1 ). Therefore, u t, z + τd) is Borel) measurable on [t k, t k+1 ) for any fixed τ and d. Since u u t, z + τd) u t, z) t, z; d) = lim, τ τ the function u t, z; d) is also Borel) measurable on [t k, t k+1 ) for any fixed z and d [3, Corollary 2.1 or Corollary 5.9]. It follows from the non-expansive property of Π Ω with respect to the Euclidean norm 2 [13] that for any given τ >, u t, z + τd) u t, z) 2 τ 1 λ τ k w i Ce At i t) B ) T 2 ft i, z + τd) ft i, z). 2 9

10 This shows that for each t [t k, t k+1 ), u t,z+τd) u t,z) 2 τ w i Lt i ) d 2 λ Ce Ati t) B) T 2, k Hence, it is easy to see that u t, z; d) is where Lt i ) > is the Lipschitz constant of fti, ). bounded on the interval [t k, t k+1 ), i.e., there exists ϱ k > such that u t, z; d) 2 ϱ k for all t [t k, t k+1 ). This shows that u t, z; d) is Lebesgue) integrable in t on [t k, t k+1 ]. In view of the above results and the Lebesgue Dominated Convergence Theorem [3, Theorem 5.6 or Corollary 5.9], we have f t, z; d) = Ce At d+ t CeAt s) Bu s, z; d)ds for all t [t k, t k+1 ]. This shows that ft, ) is directionally differentiable for each t [t k, t k+1 ]. Furthermore, since Π Ω z) Π Ω z ) 2 z z 2 for all z, z R l, and u t, z) depends on finitely many ft i, z) on the interval [t j, t j+1 ) with j = 1,..., k cf. 19)), it can be shown that for each j = 1,..., k, there exists a uniform Lipschitz constant L j > independent of t) such that for any t [t j, t j+1 ), u t, z) u t, z ) 2 L j z z 2 for all z, z R l. In view of ft, z) = Ce At z + t CeAt s) Bu s, z)ds, the continuity of f in t, and the induction hypothesis, we deduce the Lipschitz continuity of ft, ) for each fixed t [t k, t k+1 ]. Therefore, the proposition follows by the induction principle. Clearly, the assumption of global directional differentiability of the Euclidean projector Π Ω is critical to Proposition 4.1. In what follows, we identify a few important cases where this assumption holds. One of the most important cases is when Ω is polyhedral. In this case, as shown in Proposition 3.1, Π Ω ) is a continuous piecewise affine function, and its directional derivative is given by a piecewise linear function of a direction vector d cf. [13, Section 4.1] or [31]). When Ω is non-polyhedral, we consider a finitely generated convex set, i.e., Ω = {w R m Gw) }, where G : R m R p 1 is such that each component function G i is twice continuously differentiable and convex for i = 1,..., p 1. It is known that if, for each w R m, the set Ω satisfies either sequentially bounded constraint qualification SBCQ) or constant rank constraint qualification CRCQ) at Π Ω w), then Π Ω is directionally differentiable; see [13, Sections ] for detailed discussions. More differential properties can be obtained for ft, z). Motivated by [27, Theorem 8], we consider semismoothness of f. A function G : R n R m is said to be semismooth at z R n [13] if G is B-differentiable at all points in a neighborhood of z and satisfies G z; z z ) G z ; z z ) lim z z z z z Semismooth functions play an important role in nonsmooth analysis and optimization; see [13] and the references therein for details. Lemma 4.2. Assume that Π Ω : R m R m is directionally differentiable on R m. For any given {t i, y i )}, {w i }, λ >, and z R l, if u t, ) is semismooth at z for each fixed t [, 1], so is ft, ). Proof. Fix {t i, y i )}, {w i }, λ >, and z R l. It suffices to prove that xt, ) is semismooth at z for each fixed t [, 1], where xt, z) satisfies the ODE: ẋt) = Axt) + Bu t, z), t [, 1] with x) = z. It follows from the proof of Proposition 4.1 that xt, z) is B-differential in z on [, 1] and for a given d R l and t [, 1], x t, z; d) = e At d + t =. e At s) Bu s, z; d)ds. In view of this, it is easy to verify that for a fixed t [, 1] and any z R l, t ) x t, z; z z) x t, z; z z) = e At s) B u s, z; z z) u s, z; z z) ds. By the semismoothness of u s, ) at z, we have for each fixed s [, t], u lim s, z; z z) u s, z; z z) =. z z z z z 1

11 Furthermore, it is shown in the proof of Proposition 4.1 that u s, z; z z) and u s, z; z z) are Lebesgue integrable and bounded on [, 1]. Therefore, it follows from Lebesgue Dominated Convergence Theorem [3, Theorem 5.6 or Corollary 5.9] that x t, z; z z) x t, z; z z) lim =. z z z z z This shows that xt, z) is semismooth at z for each t [, 1]. Proposition 4.2. If Π Ω is semimsooth at any point in R m, then for any z R l, ft, ) is semismooth at z for each t [, 1]. In particular, this holds true if Ω is polyhedral. Proof. Note that semimsoothness implies B-differentiability. Furthermore, ft, ) is clearly semismooth in z on [, t 1 ]. Now assume that ft, z) is semismooth in z for all t [, t k ]. By the induction hypothesis and 19), we see that for any fixed t [t k, t k+1 ], u t, z) is a composition of Π Ω and a semismooth function of z. It follows from [13, Proposition 7.4.4] that u t, ) is semismooth at z for any t [t k, t k+1 ]. In light of Lemma 4.2, ft, ) is semismooth at z on [t k, t k+1 ] and on [, t k+1 ]. By the induction principle, ft, ) is semismooth at z for each t [, 1]. Finally, if Ω is polyhedral, then Π Ω is continuous piecewise affine and hence strongly) semismooth [13, Proposition 7.4.7]. It follows from the above results that H y,n is a vector-valued B-differentiable function provided that Π Ω ) is directionally differentiable). To solve the equation H y,n z) =, we consider a nonsmooth Newton s method with line search in [24]; its unique) solution is the optimal initial state x that completely determines the smoothing spline ft, x ). It is worth pointing out that the original nonsmooth Newton s method in [24] assumes the existence of a direction vector d solving the equation H y,n z) + H y,nz; d) = for any z. While this assumption is shown to be true for almost all z in Theorem 5.1, it is highly difficult to show it for certain degenerate z; we refer the reader to Section 5 for the definition of a degenerate z. To overcome this difficulty, we show in Proposition 5.2 that a suitable small perturbation to a degenerate z yields a non-degenerate vector for which the assumption is satisfied. This leads to a modified nonsmooth Newton s method for the constrained smoothing spline; we postpone the presentation of this modified algorithm to Section 5 after all essential technical results are given. Moreover, it is noted that if f is semismooth, other nonsmooth Newton s methods may be applied [13]. However, these methods require computing multiple limiting Jacobians, which is usually numerically expensive. On the other hand, the modified nonsmooth Newton s method only requires computing directional derivatives and a single Jacobian) at a non-degenerate point. Before ending this section, we show that for any given z R l, the level set S z := {z R l : H y,n z) H y,n z ) } is bounded. This boundedness property will be critical for convergence analysis of the modified nonsmooth Newton s method; see the proof of Theorem 5.2. We introduce some technical preliminaries first. Recall that the recession cone of a closed convex set K in R n is defined by K := {d R n x + µd K, µ } for some x K. It is known [1] that in a finite dimensional space such as R n, K is equivalent to the asymptotic cone of K defined by { d R n there exist < µ k, x k K such that x k lim = d k µ k Furthermore, K is a closed convex cone, and K is bounded if and only if K = {}. More equivalent definitions and properties of recession cones can be found in [1, Proposition 2.1.5]. We give a lemma pertaining to the Euclidean projection onto a recession cone as follows. }. 11

12 Lemma 4.3. Let Ω be a closed convex set in R m, let v k ) be a sequence in R m, and let µ k ) be a positive real sequence such that lim k µ k = and lim k v k µ k = d for some d R m. Then where Ω is the recession cone of Ω. Π Ω v k ) lim = Π Ω d), k µ k Proof. It follows from a similar argument as in [13, Lemma ] that Π Ω µd) lim µ µ = Π Ω d). Π Therefore, it suffices to show lim Ω v k ) Π k µ k = lim Ω µ k d) k µ k. Without loss of generality, we let the vector norm be the Euclidean norm. By virtue of the non-expansive property of the Euclidean projector with respect to the Euclidean norm, we have Π Ω v k ) Π Ω µ k d) v k µ k d = v k µ k µ k d µ k, as k. This shows the equivalence of the two limits, and hence completes the proof. With the help of this lemma, we establish the following proposition. Proposition 4.3. Let Ω be a closed convex set in R m. Given any {t i, y i )} satisfying the condition H.1, {w i }, λ >, and z R l, the level set S z is bounded. Proof. We prove the boundedness of S z by contradiction. Suppose not. Then there exists a sequence z k ) in S z such that z k as k. Without loss of generality, we may assume that z k / z k ) converges to v R l with v = 1 by taking a suitable subsequence of z k ) if necessary. Define the functions f : [, 1] R l R p and ũ : [, 1] R l R m as: and ũ t, z) := ft, z) := Ce At z + t Ce At s) Bũ s, z) ds, Π Ω ), t [, t 1 ) k Π Ω λ 1 w i Ce At i t) B )T ) fti, z), t [t k, t k+1 ), k = 1,..., n 1 where Ω is the recession cone of Ω. Note that f can be treated as the shape restricted smoothing spline obtained from the linear control system ΣA, B, C, Ω ) for the given ỹ := ỹ i ) n =, namely, the control constraint set Ω is replaced by its recession cone Ω and y by the zero vector. We claim that for each fixed t [, 1], lim k ft, z k ) z k = ft, v ). We prove this claim by induction on the intervals [t j, t j+1 ] for j =, 1,..., n 1. Consider the interval [, t 1 ] first. Recall that u t, z k ) = Π Ω ), t [, t 1 ) such that ft, z k ) = Ce At z k + t CeAt s) BΠ Ω )ds for all t [, t 1 ]. Hence, in view of Π Ω ) = such that ũ t, v ) = and ft, v ) = Ce At v for all t [, t 1 ], we have, for each fixed t [, t 1 ], lim k ft, z k ) z k = lim k CeAt z k z k = CeAt v = ft, v ). 12

13 Now suppose the claim holds true for all t [, t j ] with j {1,..., n 2}, and consider [t j, t j+1 ]. Note that for each t [t j, t j+1 ), u t, z) = Π Ω λ 1 j w i Ce At i t) B ) T ) ) fti, z) y i. By the induction hypothesis and the boundedness of Ce At i t) B on [t j, t j+1 ] for all i = 1,..., j, we have, for each fixed t [t j, t j+1 ), λ 1 j lim w i Ce At i t) B ) T ) fti, z) y i k z k j = λ 1 w i Ce At i t) B )T fti, v ). By Lemma 4.3, we further have, for each fixed t [t s, t s+1 ) with s {1,..., j}, u t, z k ) Π Ω λ 1 s w i Ce At i t) B ) T ) ) fti, z) y i lim = lim k z k k z k ) s = Π Ω λ 1 w i Ce At i t) B )T fti, v ) = ũ t, v ). Clearly, ũ, v ) is Lebesgue integrable and uniformly bounded on [t j, t j+1 ]. Therefore, for each fixed t [t j, t j+1 ], lim k ft, z k ) z k Ce At z k + = lim k Ce At z k = lim k z k + = Ce At v + = ft, v ), t t t Ce At s) Bu s, z k ) ds z k Ce At s) B lim k Ce At s) Bũ s, v ) ds ) u s, z k ) ds z k where the second equality follows from the Lebesgue Dominated Convergence Theorem [3, Theorem 5.6]. This establishes the claim by the induction principle. In light of the claim and the definition of H y,n in 18), we hence have H y,n z k ) lim k z k = n )T w i Ce At i fti, v ) = Hỹ,n v ) ỹ=, where Hỹ,n : R l R l with ỹ = ỹ i ) n Hỹ,n z) := is defined by n ) w i Ce At i T fti, z) ỹ i ). Since the smoothing spline f is obtained from the linear control system ΣA, B, C, Ω ), and the recession cone Ω contains the zero vector, it is easy to verify that when ỹ =, the optimal solution pair ũ, x ) for ft, x ) is x = and ũ t, x ) on [, 1] such that ft, x ) on [, 1]). Based 13

14 on Lemma 4.1, we deduce that the equation H,n z) = has a unique solution z =. Since v, we must have H,n v ). Consequently, H y,n z k ) lim k z k = H,n v ) >. This shows that H y,n z) is unbounded on S z, which yields a contradiction. 5 The Modified Nonsmooth Newton s Method: Algorithm and Convergence Analysis In this section, we study the modified nonsmooth Newton s method and its global convergence. In particular, we focus on the case where the control constraint set Ω is polyhedral for the following reasons: i) the class of polyhedral Ω is already very broad and includes a number of important applications; ii) since any closed convex set is the intersection of all closed half-spaces containing it, such a set can be approximated by a polyhedron with good precision; iii) when Ω is polyhedral, Π Ω is globally B-differentiable, while this is not the case for a non-polyhedral Ω, unless certain constraint qualifications are imposed globally. Furthermore, for a non-polyhedral Ω, the directional derivatives of Π Ω are much more difficult to characterize and compute. Let Ω = {w R m Dw b} be a polyhedron with D R r m and b R r. Proposition 3.1 shows that Π Ω : R m R m is a Lipschitz) continuous and piecewise affine PA) function. It follows from 19) that for t [t k, t k+1 ) with k = 1, 2,..., n 1, Bu t, z) = BΠ Ω B T e AT t v k z) ), where v k z) := λ 1 k ) ) w i Ce At i T fti, z) y i R l. 2) Define the function F : R l R l as F := B Π Ω B T, which is also Lipschitz continuous and piecewise affine. It follows from the theory of piecewise smooth functions e.g., [31]) that such a function admits an appealing geometric structure for its domain, which provides an alternative representation of the function. Specifically, let Ξ be a finite family of polyhedra {X i } m, where each X i := { v R l G i v h i } for a matrix G i and a vector h i. We call Ξ a polyhedral subdivision of R l [13, 31] if a) the union of all polyhedra in Ξ is equal to R l, i.e., m X i = R l, b) each polyhedron in Ξ has a nonempty interior thus is of dimension l), and c) the intersection of any two polyhedra in Ξ is either empty or a common proper face of both polyhedra, i.e., X i X j = [ X i X j = X i {v G i v h i ) α = } = X j {v G j v h j ) β = } for nonempty index sets α and β with X i {v G i v h i ) α = } = X i and X j {v G j v h j ) β = } X j ]. For a Lipschitz PA function F, one can always find a polyhedral subdivision of R l and finitely many affine functions g i v) E i v +l i such that F coincides with one of the g i s on each polyhedron in Ξ [13, Proposition 4.2.1] or [31]. Therefore, an alternative representation of F is given by F v) = E i v + l i, v X i, i = 1,..., m, and v X i X j = E i v + l i = E j v + l j. Given v R l, define the index set Iv) := {i v X i }. Moreover, given a direction vector d R l, there exists j Iv) dependent on d) such that F v; d) = E j d. A more precise characterization 14

15 of the directional derivative of the Euclidean projection is defined by the critical cone [13, Theorem 4.1.1], which shows that for a fixed v, F v; d) is continuous and piecewise linear PL) in d.) In view of this and 19), we have that, for each fixed t [t k, t k+1 ) with k = 1,..., n 1, there exists j Ie AT t v k z)) dependent on d) such that k Bu t, z; d) = E j e AT t v k z; d), where v k z; d) = )T λ 1 w i Ce At i f t i, z; d). 21) For each fixed t, the matrix E j not only depends on z, which is usually known, but also depends on the direction vector d that is unknown a priori in a numerical algorithm. This leads to great complexity and difficulty in solving the equation H y,n z) + H y,nz; d) = for a given z, where d is the unknown. In what follows, we identify an important case where e AT t v k z) is in the interior of some polyhedron X j such that the matrix E j relies on z and t) but is independent of d. For notational convenience, define qt, v) := e AT t v, v R l, which satisfies the linear ODE: qt, v) = A T qt, v). For a polyhedron X i = {v G i v h i } in Ξ, define Y i := {v R l G i v h i, G i A T )v, G i A T ) 2 v,..., G i A T ) l v) }, where denotes the lexicographical nonnegative order. For a given v R l, let the index set J v) := {i v Y i }. Clearly, J v) Iv) for any v. Furthermore, given a t, qt, v) X i for all t [t, t + ε] for some ε > if and only if qt, v) Y i. We introduce more concepts as follows. Definition 5.1. Let qt, v) and a time t be given. If J qt, v)) Iqt, v)), then we call t a critical time along qt, v) and its corresponding state qt, v) a critical state. Furthermore, if there exist ε > and a polyhedron X i in Ξ such that qt, v) X i, t [t ε, t + ε], then we call t a non-switching-time along qt, v); otherwise, we call t a switching time along qt, v). It is known that a switching time must be a critical time but not vice versa [32]. Furthermore, a critical state must be on the boundary of a polyhedron in Ξ. The following result, which is a direct consequence of [32, Proposition 7], presents an extension of the so-called non-zenoness of piecewise affine or linear systems e.g., [5, 25, 33, 34]). Proposition 5.1. Consider qt, v) and a compact time interval [t, t +T ] where T >. Then there are finitely many critical times on [t, t + T ] along qt, v). Particularly, there exists a partition t = t < t 1 < < t M 1 < t M = t + T such that for each i =, 1,..., M 1, Iqt, v)) = J qt, v)) = J qt, v)), t t i, t i+1 ) for any t t i, t i+1 ). It follows from the above proposition that for any given v, there are finitely many critical times on the compact time interval [t k, t k+1 ] along qt, v), where k 1. We call qt, v) non-degenerate on [t k, t k+1 ] if, for any two consecutive critical times t j and t j+1 on [t k, t k+1 ], there exists an index i dependent on t j, t j+1 )) such that Iqt, v)) = {i } for all t t j, t j+1 ). In other words, qt, v) is non-degenerate if it is in the interior of some polyhedron of Ξ on the entire t j, t j+1 ). We introduce more notation and assumptions. First, it is clear that there exist constants ρ 1 > and ρ 2 > such that Ce At s) ρ 1 for all t, s [, 1] and max i {1,...,m } E i ρ 2. In addition, we assume that H.2 there exist constants ρ t >, µ ν > such that for all n, max t i+1 t i ρ t i n 1 n, ν n w i µ n, i. 15

16 Theorem 5.1. Let Ω be a polyhedron in R m. Assume that H.1 H.2 hold and λ µ 2 ρ 2 1 ρ 2ρ t /4ν). Given z R l, let v k z) be defined as in 2). Suppose that qt, v k z)) = e AT t v k z) is nondegenerate on [t k, t k+1 ] for each k = 1, 2..., n 1. Then there exists a unique direction vector d R l satisfying H y,n z) + H y,nz; d) =. Proof. It follows from the non-degeneracy of qt, v k z)) and Proposition 5.1 that, for the given z and each [t k, t k+1 ] with k = 1,..., n 1, there exists a partition t k = t k, < t k,1 < < t k,mk 1 < t k,mk = t k+1 such that for each j =,..., M k 1, qt, v k z)) is in the interior of some polyhedron of Ξ for all t t k,j, t k,j+1 ). It is easy to show via the continuity of ft, z) in z that for each open interval t k,j, t k,j+1 ), there exists a matrix E k,j {E 1,..., E q } such that for all t t k,j, t k,j+1 ), Bu t, z; d) = E k,j e AT t v k z; d). Letting w i := w i /λ, i = 1,..., n and by 21), we have, for r k+1, tk+1 tk+1 M k 1 Ce Atr s) Bu s, z; d)ds = Ce Atr s) E k,j I [ t k,j, t k,j+1 ] e AT s v k z; d)ds t k t k j= tk+1 M k 1 k = Ce Atr s) )T E k,j I [ t k,j, t k,j+1 ] e AT s w i Ce At i f t i, z; d)ds = = t k k w i j= tk+1 Ce Atr s) t k k w i V r,k,i),z f t i, z; d), where, for each i = 1,..., k, V r,k,i),z := tk+1 Ce Atr s) t k j= j= M k 1 E k,j I [ t k,j, t k,j+1 ] Ce At i s) ) T ds f t i, z; d) M k 1 E k,j I [ t k,j, t k,j+1 ] Ce At i s) ) T ds R p p. Note that for a fixed triple r, k, i), V r,k,i),z depends on z only and r > k i 1. For r > i 1, define W r,i),z := w i r 1 j=i V r,j,i),z. Therefore, for each k = 1,..., n 1, f t k+1, z; d) = Ce At k+1 d + = Ce At k+1 d + k j=1 k j=1 tj+1 t j Ce At k+1 s) Bu s, z; d)ds j w i V k+1,j,i),z f t i, z; d) = Ce At k+1 d + k W k+1,i),z f t i, z; d). In what follows, we drop z in the subscript of W for notational simplicity. In view of f t 1, z; d) = Ce At 1 d, it can be shown via induction that for each k = 2,..., n, ) f t k, z; d) = Ce At k d + W k,k 1) Ce At k 1 d + W k,k 2) + W k,k 1) W k 1,k 2) Ce At k 2 d + + k 1 s W k,1) + W k,s) W s,1) + W k,s1 )W s1,s 2 )W s2,1) + s 1 =3 s 2 =2 + W k,k 1) W k 1,k 2) W 3,2) W 2,1) )Ce At 1 d = k W k,j) Ce At j d, j=1 16

17 where the matrices W k,j) of order p are defined in terms of W k,s) as shown above. For a given r {1,..., n}, define Ce At I p 1 Ce At 2 W 2,1) I p C r :=. Rrp l, W r := diagw 1 I p,..., w r I p ) W 3,1) W3,2) I p R rp rp, Ce Atr W r,1) Wr,2) Wr,r 1) I p 22) where I p is the identity matrix of order p and W r depends on z but is independent of d. Hence, the directional derivative of r w ice At i ) T ) fti, z) y i along the direction vector d is given by r )T w i Ce At i f t i, z; d) = C T r W r C r d. Clearly, W r is invertible for any r, and it can be easily verified via the property of W k,j) that W 1 r = I p W 2,1) I p W 3,1) W 3,2) I p W r,1) W r,2) W r,r 1) I p Moreover, define the symmetric matrix V r := 1 2 W 1 r + W 1 ) ) T r = I p w 1 W 2,1) 2w 1 W 3,1) W 2,1) 2w 1 W 3,1) 2w 1 W 3,2) 2w 2. W r,1) 2w 1 diagw 1 1 I p,..., w 1 r I p ). 2w 1 W r,1) 2w 1 I p w 2 W 3,2) 2w 2 W r,2). 2w 2 I p... w W r,r 1) 2w r 1 W r,2) 2w 2 W r,r 1) 2w r 1 It follows from assumption H.2 that max i w i µ/λn) and that for any 1 j < k, k 1 w k W k,i) + 2w i W k,j) µ tk ρ 2 λ n 1ρ 2 dτ µρ2 1 ρ 2 t j λ n ρtk j). n w Furthermore, we deduce from H.2 that max i i,j w j µ/ν. Therefore, for any fixed k = 1,..., n, ) n k 1 ) n i=k+1 W i,k) 2w k µ2 ρ 2 1 ρ 2ρ t 2λν n 2 µ2 ρ 2 1 ρ 2ρ t 2λν n 2 n 1 k i) + i=k+1 I p w r i k) i µ2 ρ 2 1 ρ 2ρ t n 1) 4λν n where the last inequality follows from the assumption on λ. This implies that the symmetric matrix V r is strictly diagonally dominant for any r. Hence, for each r, V r is positive definite, so are W 1 r and W r although not symmetric). Finally, note that H y,nz; d) = C T n W n C n d, and C n has full column rank, in light of the assumption H.1. Consequently, C T n W n C n is positive definite such that the unique direction vector d = C T ) 1Hy,n n W n C n z) solves the equation H y,n z) + H y,nz; d) =. 17 < 1,

18 The above result relies on the critical non-degenerate property of qt, v k z)). In what follows, we consider the case where qt, v k z)) is degenerate on some sub-interval of [t k, t k+1 ]. Geometrically, this implies that the trajectory of qt, v k z)) travels on a face of a polyhedron in Ξ for some time. It shall be shown that under mild assumptions, a suitable small perturbation of z will lead to a nondegenerate trajectory. Recall that each polyhedron X i in the polyhedral subdivision Ξ is defined by the matrix G i R m i and the vector h i R m i. Since each X i has non-empty interior, we assume, without loss of generality, that for each j = 1,..., m i, the set {v X i G i v h i ) j = } represents a unique) facet of X i i.e., a l 1)-dimensional face of X i ) [31, Proposition 2.1.3], where G i ) j denotes the jth row of G i and satisfies G i ) T j 2 = 1. Proposition 5.2. Let Ω be a polyhedron in R m. For a given z R l, suppose that qt, v k z)) is degenerate on the interval [t k, t k+1 ] for some k {1,..., n 1}, where v k z) is defined in 2). Assume that C, A) is an observable pair, H.1 H.2 hold, and λ µ 2 ρ 2 1 ρ 2ρ t /4ν). Then for any ε >, there exists d R l with < d ε such that qt, v k z + d)) is non-degenerate on [t k, t k+1 ] for each k = 1,..., n 1. Proof. Fix ε >. Define the set of vector-scalar pairs that represent all the facets of the polyhedra in Ξ: { Gi S := ) T ) } j, h i ) j i = 1,..., m, j = 1,..., m i. Note that if qt, v k z)) is degenerate on [t k, t k+1 ] for some k, then there exist a pair g, α) S and an open subinterval T [t k, t k+1 ] such that g T qt, v k z)) α = for all t T, which is further equivalent to g T qt, v k z)) α = for all t [t k, t k+1 ] in view of qt, v k z)) = e AT t v k z). Therefore, we define for each k {1,..., n 1}, S z,k,d := {g, α) S g T qt, v k z)) = α, t [t k, t k+1 ]}. Let k 1 be the smallest k such that S z,k,d is nonempty or equivalently qt, v k z)) is degenerate on [t k, t k+1 ]). Clearly, k 1. Since qt, v k z)) is non-degenerate on [t k, t k+1 ] for each k = 1,..., k 1 1, it follows from a similar argument in the proof of Theorem 5.1 that v k 1 z; d) = λ 1 C T k 1 W k1,z C k1 d, where we write W k1 as W k1,z to emphasize its dependence on z but independent of d). Consider the following two cases: i) g, α) S z,k1,d. It follows from the B-differentiability of v k1 ) that qt, v k1 z + d)) = qt, v k1 z)) + q t, v k 1 z; d) + o d ) ) for each t [t k1, t k1 +1] [13, Proposition 3.1.3]. Therefore, using the fact that g 2 = 1, we have for each t [t k1, t k1 +1], g T qt, v k1 z+d)) α = g T qt, v k1 z)) α ) +g T q t, v k 1 z; d)+o d ) ) = g T q t, v k 1 z; d) ) +o d ). Furthermore, under H.2 and the assumption on λ, it is shown in Theorem 5.1 that W k1,z is positive definite. Let the observability matrix g T g T A T V g :=. Rl l. g T A T ) l 1 Since g T qt, v k 1 z; d)) = g T e AT t C T k 1 W k1,z C k2 d, we see that g T qt, v k 1 z; d)) is nonvanishing on [t k1, t k1 +1] if and only if d / KerV g C T k 1 W k1,z C k1 ). Since g is nonzero, W k1,z is positive definite, and C, A) is an observable pair, it is easy to show that V g C T k 1 W k1,z C k1 such that KerV g C T k 1 W k1,z C k1 ) is a proper subspace of R l. Hence there exists a scalar τ > such that for any d / KerV g C T k 1 W k1,z C k1 ) with < d τ, g T qt, v k 1 z; d)), and thus g T qt, v k1 z + d)) α, is nonvanishing on [t k1, t k1 +1], which further implies that g T qt, v k1 z + d)) α has at most finitely many zeros on [t k1, t k1 +1]. 18