MORDUKHOVICH SUBGRADIENTS OF THE VALUE FUNCTION IN A PARAMETRIC OPTIMAL CONTROL PROBLEM. Nguyen Thi Toan. L ( t, x(t),u(t),θ(t) ) dt

Transcription

1 TAIWANESE JOURNAL OF MATHEMATICS Vol. 19, No. 4, pp , August 215 DOI: /tjm This paper is available online at MORDUKHOVICH SUBGRADIENTS OF THE VALUE FUNCTION IN A PARAMETRIC OPTIMAL CONTROL PROBLEM Nguyen Thi Toan Abstract. This paper is devoted to the study the first-order behavior of the value function of a parametric optimal control problem with linear constraints and a nonconvex cost function. By establishing an abstract result on the Mordukhovich subdifferential of the value function of a parametric mathematical programming problem, we derive a formula for computing the Mordukhovich subdifferential of the value function to a parametric optimal control problem. 1. INTRODUCTION A wide variety of problems in optimal control problem can be posed in the following form. Determine a control vector u L p [, 1], R m ) and a trajectory x W 1,p [, 1], R n ), 1 <p<, which minimize the cost 1) g x1) ) + L t, xt),ut),θt) ) dt with the state equation 2) ẋt) =At)xt)+Bt)ut)+T t)θt) a.e. t [, 1], and the initial value 3) x) = α. Here W 1,p [, 1], R n ) is the Sobolev space, which consists of absolutely continuous functions x :[, 1] R n such that ẋ L p [, 1], R n ). Its norm is given by x 1,p = x) + ẋ p. Received August 5, 213, accepted October 21, 214. Communicated by Jen-Chih Yao. 21 Mathematics Subject Classification: 49J21, 49K21, 93C5. Key words and phrases: Parametric optimal control problem, Marginal function, Value function, Mordukhovich normal cone, Mordukhovich subgradients, Mordukhovich subdifferential. The author s research was partially supported by the NAFOSTED of National Foundation for Science & Technology Development Vietnam). 151

2 152 Nguyen Thi Toan are matrix- The notations in 1) 3) have the following meanings: x, u are the state variable and the control variable, respectively, α, θ) R n L p [, 1], R k ) are parameters, g : R n R, L :[, 1] R n R m R k R are given functions, At) = a ij t) ) n n, Bt) = b ij t) ) n m and T t) = c ij t) ) n k valued functions. Put X = W 1,p [, 1], R n ), U = L p [, 1], R m ), Z = X U, Θ=L p [, 1], R k ), W = R n Θ. It is well known that X, U, Z, Θ and W are Asplund spaces. For each w =α, θ) W, the optimal value and the solution set of the problem 1)-3) corresponding to parameter w W are denoted by V w) and Sw), respectively. Thus, V : W R is an extended real-valued function, which is called the value function or the marginal function of the problem 1) 3). It is assumed that V is finite at w and z = x, ū) is a solution of the problem corresponding to a parameter w, thatis z = x, ū) S w). The study of the first-order behavior of value functions is important in variational analysis and optimization. An example is the study of distance functions and its applications to optimal control problems see [2, 1, 28]). There are many papers dealing with differentiability properties and the Fréchet subdifferential of value functions see [8, 9, 15, 16, 17, 2]). By considering a set of assumptions, which involves a kind of coherence property, Penot [2] showed that the value functions are Fréchet differentiable. The results of Penot gave sufficient conditions under which the value functions are Fréchet differentiable rather than formulas for computing their derivatives. In [17], Mordukhovich, Nam and Yen derived formulas for computing and estimating the Fréchet subdifferential and Mordukhovich subdifferential of value functions of parametric mathematical programming problems in Banach spaces. Besides the study of the first-order behavior of value functions in parametric mathematical programming, the study of the first-order behavior of value functions in optimal control problems has attracted attention of many researchers see [4, 6, 7, 12, 18, 19, 21, 22, 23, 24, 25, 26] and [27]). Recently, Toan and Kien [25] have derived a formula for an upper evaluation of the Fréchet subdifferential of the value function V for the case where g and L were not assumed to be convex. Under some assumptions which are weaker than the ones in [25], Chieu, Kien and Toan in [6] have obtained a formula for an upper evaluation of the Fréchet subdifferential of the value function V,which complements the results in [25]. However, we still could not find a suitable upper evaluation for the Mordukhovich subdifferential of the value function V in the case g and L not assumed to be convex.

3 Mordukhovich Subgradients of the Value Function in a Parametric Optimal Control Problem 153 It is known that Mordukhovich subdifferential is the outer limit of a family of the ε- Fréchet subdifferentials. Hence, upper estimates for the Mordukhovich subdifferential of the value function V are more complicated than the corresponding results in [6] and [25]. The aim of this paper is to derive some new formulas for computing the Mordukhovich subdifferential of V at w via the tool of generalized differentiation. In order to obtain the result, we first establish a formula for computing and estimating the Mordukhovich differential of value functions for a special class of parametric mathematical programming problems. We then apply the obtained result to our problem together with using some techniques of functional analysis. We note that upper estimates for the Fréchet subdifferential in [6] are at the same time upper estimates for the Mordukhovich subdifferential of the value function V if the solution map S is V -inner semicontinuous at w, z). Besides, the Mordukhovich subdifferential of the value function V at w contains the Fréchet subdifferential of V at w. So, the upper-evaluation in this paper is better than the corresponding evaluation in [6]. Let us recall some notions on generalized differentiation, which are related to our problem. The notions and results of generalized differentiation can be found in [3], [13] and [14]. Let Z be an Asplund space, ϕ : Z R be an extended real-valued function and z Z be such that ϕ z) is finite. For each ε, theset ˆ ε ϕ z) := z Z ϕz) ϕ z) z,z z } lim inf ε z z z z is called the ε-fréchet subdifferential of ϕ at z. A given vector z ε ϕ z) is called an ε-fréchet subgradient of ϕ at z. The set ϕ z) = ϕ z) is called the Fréchet subdifferential of ϕ at z and the set 4) ϕ z) :=Limsup ϕ z z ε ε ϕz) is called the Mordukhovich subdifferential of ϕ at z, where the notation z ϕ z means z z and ϕz) ϕ z). Hence z ϕ z) there exist sequences z k ϕ z, εk +, and z k εk ϕz k ) w such that zk z.ifϕis lower semicontinuous around z, then we can equivalently put ε = in 4). Moreover, we have ϕ z) for every locally Lipschitzian function. It is known that the Mordukhovich subdifferential reduces to the classical Fréchet derivative for strictly differentiable functions and to subdifferential of convex analysis for convex functions. The set ϕ z) :=Limsupλ ε ϕz) ϕ z z λ,ε

4 154 Nguyen Thi Toan is called the singular subdifferential of ϕ at z. Hence z ϕ z) there exist sequences z k ϕ z, εk +,λ k +, and z k λ k εk ϕz k ) such that z k Let Ω be a nonempty set in Z and z Ω. Theset N ε z ;Ω):= z Z lim sup z Ω z is called the ε-fréchet normal cone to Ω at z and the set w z. z,z z z z ε } Nz ; Ω) := lim Ω z z ε N ε z;ω) is called the Mordukhovich normal cone to Ω at z. It is also known that if Ω is a convex set, then the Mordukhovich normal cone coincides with the Fréchet normal cone and coincides with normal cone of convex analysis for convex sets. AsetΩ is called sequentially normally compact SNC) at z if for any sequences Ω ε k,z k z, andz k N εk z k ;Ω)one has [ z w k ] = [ zk ] as k, where ε k can be omitted if Ω is locally closed around z. An extended real-valued function ϕ on Z is called sequentially normally epi-compact SNEC) at z if its epigraph is SNC at z, ϕ z)). We say that a set-valued map F : Z E, wherez and E are Asplund spaces, admits a locally upper Lipschitzian selection at z, v) gphf := z, v) Z E v F z) } if there is a single-valued mapping φ : Z E, which is locally upper Lipschitzian at z, that is there exist numbers η> and l> such that φz) φ z) l z z whenever z B z, η), which satisfies φ z) = v and φz) F z) for all z in a neighborhood of z. We now return to the problem 1) 3). For each w =α, θ) W, we put and 5) Jx, u, w)=g x1) ) + L t, xt),ut),θt) ) dt Gw) = z =x, u) X U 2) and 3) are satisfied }.

5 Mordukhovich Subgradients of the Value Function in a Parametric Optimal Control Problem 155 Then the problem 1) 3) can be formulated in the following form: 6) V w) = inf Jz, w). z Gw) We say that the solution map Sw) is V -inner semicontinuous at w, z) if for every V sequence w k w i.e., wk w and V w k ) V w) ), there is a sequence z k } with z k Sw k ) for all k, which contains a subsequence converging to z. To deal with our problem, we impose the following assumptions: H1) The functions L :[, 1] R n R m R k R and g : R n R have properties that L,x,u,v) is measurable for all x, u, v) R n R m R k, Lt,,, ) and g ) are continuously differentiable for almost every t [, 1], andthereexist constants c 1 >,c 2 >,r, a nonnegative function ω 1 L p [, 1], R), constants p 1 p, p 2 p 1 such that Lt, x, u, v) c 1 ω1 t)+ x p 1 + u p 1 + v p ) 1, max L x t, x, u, v), L u t, x, u, v), L v t, x, u, v) } c 2 x p 2 + u p 2 + v p ) 2 +r for all t, x, u, v) [, 1] R n R m R k. H2) The matrix-valued functions A :[, 1] M n,n R), B :[, 1] M n,m R) and T :[, 1] M n,k R) are measurable and essentially bounded. H3) There exists a constant c 3 > such that T T t)v c 3 v v R n, a.e. t [, 1]. We are now ready to state our main result. Theorem 1.1. Suppose that the solution map S is V -inner semicontinuous at w, z) gphs, and assumptions H1) H3) are fulfilled. Then for a vector α,θ ) R n L q [, 1], R k ) to be a Mordukhovich subgradient of V at ᾱ, θ), it is necessary that there exists a function y W 1,q [, 1], R n ) such that the following conditions are satisfied: α = g x1) ) ) 1 7) + L x t, xt), ūt), θt) dt A T t)yt)dt, 8) y1) = g x1) ) and 9) ) ẏt)+a T t)yt),b T t)yt),θ t)+t T t)yt) = L t, xt), ūt), θt) ) a.e. t [, 1].

6 156 Nguyen Thi Toan The above conditions are also sufficient for α,θ ) V ᾱ, θ) if the solution map S has a locally upper Lipschitzian selection at w, x, ū). Here, A T stands for the transpose of A, L t, xt), ūt), θt) ) ) stands for the gradient of Lt,,, ) at xt), ūt), θt) and q is the conjugate number of p, that is, 1 <q<+ and 1/p +1/q =1. In order to prove Theorem 1.1 we first reduce the problem 1) 3) to a mathematical programming problem and then establish some formulas for computing the Mordukhovich subdifferential of its value function. This procedure is presented in Section 2. A complete proof for Theorem 1.1 will be provided in Section THE OPTIMAL CONTROL PROBLEM AS A PROGRAMMING PROBLEM In this section, we suppose that X, W and Z are Asplund spaces with the dual spaces X, W and Z, respectively. Assume that M : Z X and T : W X are continuous linear mappings. Let M : X Z and T : X W be adjoint mappings of M and T, respectively. Let function f : W Z R be lower semicontinuous around w, z). For each w W, we put Hw):= z Z Mz = Tw }. Consider the problem of computing the Mordukhovich subdifferential of the value function 1) hw) := inf fw, z). z Hw) This is an abstract model for 6). We denote by Ŝw) the solution set of 1) corresponding to parameter w W. Assume that the value function h is finite at w and z is a solution of the problem corresponding to a parameter w, thatis z Ŝ w). We now derive a formula for computing the Mordukhovich subdifferential and the singular subdifferential of the value function to the parametric programming problem 1). Note that the structure of the formulas for evaluation of the Mordukhovich subdifferential h w) and singular subdifferential h w) is different from although having some similarities) those established in [6]. The main differences are that we do not assume + f w, z) and instead of using the intersection over w,z ) + f w, z) as in [6, Theorem 3.1], we use the union over w,z ) f w, z) or w,z ) f w, z) to evaluate h w) and h w). On the other hand, we need additional assumptions that the objective function f is SNEC at w, z) and the solution map Ŝ is h-inner semicontinuous at w, z). Theorem 2.1. Suppose the solution map Ŝ is h-inner semicontinuous at w, z) gphŝ, f is SNEC at w, z) and

7 Mordukhovich Subgradients of the Value Function in a Parametric Optimal Control Problem 157 i) the following qualification condition is satisfied [ f w, z) ] [ T x, M x ):x X }] = }, ii) there exists a constant c> such that Then one has T x c x, x X. 11) 12) h w) h w) w,z ) f w, z) w,z ) f w, z) [ w + T M ) 1 z ) )]. [ w + T M ) 1 z ) )]. Moreover, assume that f is strictly differentiable at w, z) and the solution map Ŝ admits a local upper Lipschitzian selection at w, z). Then 13) h w) = w f w, z)+t M ) 1 z f w, z)) ). For the proof of this theorem, we need the following lemma from [6]. Here we put Q =gphh. Lemma 2.2. Suppose that assumptions of Theorem 2.1 are satisfied. Then for each w, z) Q, one has N w, z); Q ) = T x,m x ) x X }. We now consider the mapping ϑ : W Z R defined by setting ϑw, z)=fw, z)+δ w, z); Q ), where δ ; Q ) is the indicator function of Q, thatisδ w, z); Q ) =if w, z) Q and δ w, z); Q ) =+ otherwise. Proof of Theorem 2.1. Take any w h h w). Then there exist sequences w k w, ε k +,andwk ˆ εk hw k ) such that wk w w as k. Hence, there is a sequence η k such that So w k,w w k hw) hw k )+2ε k w w k, w w k + η k B W.

8 158 Nguyen Thi Toan w k,w w k +,z z k ϑw, z) ϑw k,z k )+2ε k w w k + z z k ), for all z k Ŝw k), w, z) w k,z k )+η k B W Z, k N. This gives wk, ) ˆ εk ϑw k,z k ) with ε k =2ε k. Since Ŝ is h-inner semicontinuous at w, z), we can select a sequence of z k Ŝw k) converging to z. Observe that ϑw k,z k ) ϑ w, z) due to hw k ) h w). Hence h w) w W :w, ) ϑ w, z) } = w W :w, ) [ fw, z)+δ w, z); Q )] w, z) }. From the assumed SNEC property of f at w, z), condition i) and [13, Theorem 3.36], it follows that h w) w W :w, ) f w, z)+n w, z); Q )}, which is equivalent to h w) w W :w, ) w 1,z 1 ) f w, z) By Lemma 2.2, there exists x X such that for some w1,z 1 ) f w, z). Hence Consequently w w 1 = T x ) and z 1 = M x ), [ w 1,z 1)+N w, z); Q )]}. w = w 1 + T x ) and x =M ) 1 z 1 ). w w 1 + T [ M ) 1 z 1) ], for some w1,z 1 ) f w, z). We obtain the inclusion 11). To prove the inclusion 12), take any w h w). Then there exist sequences h w k w, εk +,ξ k +,andwk ξ k ˆ εk hw k ) such that wk w w as k. So ξ 1 k wk ˆ εk hw k ), k N. Hence, there is a sequence η k such that So 1 ξ k w k,w w k hw) hw k )+2ε k w w k, w w k + η k B W. 1 ξ k w k,w w k +,z z k ϑw, z) ϑw k,z k )+2ε k w w k + z z k ),

9 Mordukhovich Subgradients of the Value Function in a Parametric Optimal Control Problem 159 for all z k Ŝw k ), w, z) w k,z k )+η k B W Z, k N. This gives ξ 1 k wk, ) ˆ εk ϑw k,z k ) with ε k = 2ε k. So wk, ) ξ k ˆ εk ϑw k,z k ). Since Ŝ is h-inner semicontinuous at w, z), we can select a sequence of z k Ŝw k) converging to z. Observe that ϑw k,z k ) ϑ w, z) due to hw k ) h w). Hence h w) w W :w, ) ϑ w, z) } = w W :w, ) [ fw, z)+δ w, z); Q )] w, z) }. From the assumed SNEC property of f at w, z), condition i), [13, Theorem 3.36] and [13, Prosition 1.79], it follows that h w) w W :w, ) f w, z)+n w, z); Q )}. By using similar arguments, we can show that w w 2 + T [ M ) 1 z 2) ], for some w 2,z 2 ) f w, z). Hence, we obtain the first assertion. In order to prove the second assertion, we will prove h w) [ w f w, z)+t M ) 1 z f w, z)) )]. Since ˆ h w) h w), it is sufficient to show that ˆ h w) [ w f w, z)+t M ) 1 z f w, z)) )]. On the contrary, suppose that there exists w W such that 14) w [ w f w, z)+t M ) 1 z f w, z)) )] \ ˆ h w). Then we can find γ > and a sequence w k ω such that 15) w,w k w >hw k ) h w)+ γ w k w, k N. Let φ be an upper Lipschitzian selection of the solution map Ŝ. Putting z k = φw k ), we have z k Ŝw k ) and z k z l w k w for k> sufficiently large. Hence, 15) implies w,w k w >hw k ) h w)+ γ w k w = fw k,z k ) f w, z)+ γ w k w = z f w, z),z k z + w f w, z),w k w + z k z + w k w )+ γ w k w z f w, z),z k z + w f w, z),w k w + z k z + w k w )+ γ 2 w k w + γ 2l z k z.

10 16 Nguyen Thi Toan Putting ˆγ =min γ 2, γ 2l }, we get from the above inequality that w w f w, z), z f w, z)), w k w, z k z) > z k z + w k w )+ˆγ z k z + w k w ). Consequently, lim sup w,z) Q w, z) w w f w, z), z f w, z) ), w w, z z) z z + w w ˆγ, which means that w w f w, z), z f w, z) ) / N w, z); Q ). By Lemma 2.2, we have 16) w w f w, z), z f w, z) ) / N w, z); Q ) = T x,m x ) x X }. From 14), w w f w, z) T M ) 1 z f w, z)) ). So there exists x M ) 1 z f w, z) ) such that w w f w, z) =T x ). Hence w w f w, z) = T x ) and z f w, z) =M x ). Consequently, w w f w, z), z f w, z) ) N w, z); Q ) = T v,m v ) v X }, which contradicts 16). Thus, the second assertion is valid and the proof of the theorem is complete. The next example demonstrates that the inclusion 11) of the Theorem 2.1 may hold as equality with no strict differentiability assumption on the objective function f as in 13). Example 2.1 Let Z = R 3, W = R 2, fw, z)= z w 1 z 2 w1 3 w 2

11 Mordukhovich Subgradients of the Value Function in a Parametric Optimal Control Problem 161 and Hw)= z 1,z 2,z 3 ) R 3 } : z 1 + z 2 =2w 1,z 3 =2w 2. Assume that w =1, } ). Then assumptions of Theorem 2.1 are satisfied and h w) = 3 4, 1); 3 4, 1). Besides, ˆ h w) =. Indeed, for w =1, ) we have h w) = inf z1 3 +3z 2 1 }, z 1,z 2,z 3 ) H w) where H w) = z 1,z 2,z 3 ) R 3 : z 1 + z 2 =2,z 3 = }. It is easy to check that z =1, 1, ) is the unique solution of the problem corresponding to w and therefore h w) =1. By a direct computation, we see that Ŝw)= w1, 2w 1 )} w 1, 2w 2, where w =w 1,w 2 ). Thus, the solution map Ŝ : R2 R 3 is h-inner simicontinuous at w, z). It is easy to check that f is locally Lipschitzian around w, z). By [13, Corollary 1.81], we have f w, z) =}. So, condition i) of Theorem 2.1 is satisfied. Note that 1 [ ] M = 1 and T 2 = 2 1 from which it follows that condition ii) of Theorem 2.1 is satisfied. It is known that if gw)= w, then g) = 1, 1} and ˆ g) =. Hence, 11) becomes 17) = h w) w,z ) f w, x) [ w + T M ) 1 z ) )] 9 4, 1 ) +T M ) 1 3 4, 3 4, )) ; 9 4, 1 ) +T M ) 1 3 4, 3 4, ))}. We have M ) 1 3 4, 3 4, ) = 3 4, ) and T M ) 1 } 3 4, 3 4, )) = 3 2, ). Combining this with 17), we get h w) 3 4, 1); 3 4, 1). By computing directly, we see } that h w) = 3 4, 1); 3 4, 1) and ˆ h w) =. The next example demonstrates that the upper Lipschitzian assumption of Theorem 2.1 is sufficient but not necessary to ensure the equality in the Mordukhovich subgradient inclusion 13) for value functions. Example 2.2 Let Z = R 3, W = R 2, fw, z)=w 1 z 2 1) 2

12 162 Nguyen Thi Toan and Hw) = z 1,z 2,z 3 ) R 3 : z 2 =2w 1,z 3 =2w 2 }. Assume that w =, ). Then assumptions of Theorem 2.1 are satisfied and h w) =, ) } but the solution map Ŝ ) does not admit an upper Lipschitzian selection. Indeed, for w =, ) we have h w) = inf z 1,z 2,z 3 ) H w) z4 1, where H w) = z 1,z 2,z 3 ) R 3 : z 2 =,z 3 = }. It is easy to check that z =,, ) is the unique solution of the problem corresponding to w and therefore h w) =. By a direct computation, we see that w1, 2w 1, 2w 2 ), w 1, 2w 1, 2w 2 ) } if w 1 > Ŝw)=, 2w1, 2w 2 ) } if w 1, where w =w 1,w 2 ). Thus, the solution map Ŝ : R2 R 3 is h-inner simicontinuous at w, z). But Ŝ has no locally upper Lipschitzian selection at w, z). It is easy to check that f is locally Lipschitzian around w, z). By [13, Corollary 1.81], we have f w, z) =}. So, condition i) of Theorem 2.1 is satisfied. Note that M = 1 and T = 1 [ ] 2 2 from which it follows that condition ii) of Theorem 2.1 is satisfied. Hence, 11) becomes h w) [ w f z, w)+t M ) 1 z f z, w)) )] [, ) + T M ) 1,, ) )]. Thus, we get h w), ) }. By computing directly, we see that w1 2 if w 1 hw) = if w 1 >, Hence, we have h w) =, )}. 3. PROOF OF THE MAIN RESULT To prove Theorem 1.1, we first formulate problem 1) 3) in the form to which Theorem 2.1 can be applied to. We now consider the following linear mappings:

13 Mordukhovich Subgradients of the Value Function in a Parametric Optimal Control Problem 163 A : X X defined by B : U X defined by M : X U X defined by 18) and T : W X defined by 19) Ax := x Bu := ) ) Aτ)xτ)dτ, Bτ)uτ)dτ, Mx, u):=ax + Bu T α, θ) :=α + ) T τ)θτ)dτ. Under the hypotheses H 2 ) and H 3 ), 5) can be written in the form ) ) Gw) = x, u) X U x = α + Axdτ + Budτ + ) = x, u) X U x Axdτ } = x, u) X U Mx, u)=t w). ) Budτ = α + ) ) Recall that for 1 <p<, wehavel p [, 1], R n ) = L q [, 1], R n ),where 1 <q<+, 1/p +1/q =1. Besides, L p [, 1], R n ) is pared with L q [, 1], R n ) by the formula x,x = x t)xt)dt, } Tθdτ } Tθdτ for all x L q [, 1], R n ) and x L p [, 1], R n ). Also, we have W 1,p [, 1], R n ) = R n L q [, 1], R n ) and W 1,p [, 1], R n ) is pared with R n L q [, 1], R n ) by the formula a, u),x = a, x) + ut)ẋt)dt, for all a, u) R n L q [, 1], R n ) and x W 1,p [, 1], R n ) see [11, p. 21]). In the case of p =2, W 1,2 [, 1], R n ) becomes a Hilbert space with the inner product given by x, y = x),y) + for all x, y W 1,2 [, 1], R n ). In the sequel, we shall need the following lemmas. ẋt)ẏt)dt,

14 164 Nguyen Thi Toan Lemma 3.1. [25, Lemma 2.3]). Suppose that M and T are adjoint mappings of M and T, respectively. Then the following assertions are valid: a) The mappings M and T are continuous. b) T a, u) = a, T T u ) for all a, u) R n L q [, 1], R n ). c) M a, u)= A a, u), B a, u) ),whereb a, u)= B T u and A a, u) = a A T t)ut)dt; u + for all a, u) R n L q [, 1], R n ). ) A T τ)uτ)dτ Recall that Z = X U and Gw) = x, u) X U Mx, u)=t w) }. Then our problem can be written in the form V w) := inf Jz, w) z Gw) with z =x, u) Z, w =α, θ) W and Gw) = z Z : Mz) =T w) }, ) A T t)ut)dt, where M : Z X and T : W X are defined by 18) and 19), respectively. Lemma 3.2. [25, Lemma 3.1]) Suppose that assumptions H 1 ), H 2 ) and H 3 ) are valid. Then the following assertions are fulfilled: a) There exists a constant c> such that T x c x, x X. b) The functional J is strictly differentiable at z, w) and J z, w) is given by ) w J z, w) =,L θ t, xt), ūt), θt) ), z J z, w) = J x x, ū, θ),j u x, ū, θ) ) with J u x, ū, θ) =L u, x, ū, θ) and J x x, ū, θ) = g x1) ) ) + L x t, xt), ūt), θt) dt, g x1) ) + ) L x t, xt), ūt), θt) ) dt ).

15 Mordukhovich Subgradients of the Value Function in a Parametric Optimal Control Problem 165 We now return to the proof of Theorem 1.1, our main result. Since Jx, u, w) is strictly differentiable at x, ū, w), it is also locally Lipschitzian around x, ū, w) see [13, p. 19]). By [13, Corollary 1.81], we have J x, ū, w) = }. So qualification condition i) of Theorem 2.1 is satisfied. From [13, p. 121], it follows that J is SNEC at x, ū, w). By Lemma 3.2, all conditions of Theorem 2.1 are fulfilled. According to Theorem 2.1, we obtain 2) V w) w J z, w)+t M ) 1 z J z, w)) ). We now take α,θ ) V w). By 2), it follows that α,θ ) w J z, w) T M ) 1 z J z, w)) ), which is equivalent to α,θ J θ z, w) ) T M ) 1 z J z, w)) ). Hence, there exists a, v) R n L q [, 1], R n ) such that α,θ J θ z, w) ) = T a, v) and z J z, w) =M a, v). 21) By Lemma 3.1, we get α = a; θ J θ z, w) =T T )v ) 21) Jx x, ū, w),j u x, ū, w) ) = A a, v), B a, v) ). α = a θ ) = L θ, x ), ū ), θ ) + T T )v ) g x1) ) ) 1 + L x t, xt), ūt), θt) dt = a A T t)vt)dt g x1) ) ) + L x τ, xτ), ūτ), θτ) dτ = v )+ ) ) A T τ)vτ)dτ A T t)vt)dt ) L u, x ), ū ), θ ) = B T )v ). α = a θ ) = L θ, x ), ū ), θ ) + T T )v ) g x1) ) ) 1 + L x t, xt), ūt), θt) dt = a A T t)vt)dt g x1) ) ) ) ) L x τ, xτ), ūτ), θτ) dτ = v )+ A T τ)vτ)dτ 1 ) L u, x ), ū ), θ ) = B T )v ). 1

16 166 Nguyen Thi Toan v W 1,q [, 1],R n ) θ T T ) )v )=L θ, x ), ū ), θ ) α = g x1) ) + 22) v1) = g x1) ) ) 1 L x t, xt), ūt), θt) dt + A T t)vt)dt v ) A T ) )v )=L x, x ), ū ), θ ) B T ) )v )=L u, x ), ū ), θ ). Putting y = v, wehave α = g x1) ) ) 1 + L x t, xt), ūt), θt) dt A T t)yt)dt 22) y1) = g x1) ) ẏt)+a T t)yt),b T t)yt),θ t)+t T t)yt) ) = L t, xt), ūt), θt) ), for a.e. t [, 1]. This is the first assertion of the theorem. Using the second conclusion of Theorem 2.1, we also obtain the second assertion of the theorem. The proof is complete. Example 3.1. We will illustrate the obtained result by a concrete problem. Put X = W 1,2 [, 1], R 2 ), U = L 2 [, 1], R 2 ), Consider the problem Θ=L 2 [, 1], R 2 ), W = R 2 Θ. Jx, u, θ) = x 2 1) + u u 2 + u θ ) θ2 dt inf 1 ẋ 1 =2x 1 + u 1 + θ 1 ẋ 2 = x 2 + u 2 + θ 2 23) subject to x 1 ) = α 1 x 2 ) = α 2. Let ᾱ, θ) = 1, 1),, ) ). The following assertions are valid: i) The pair x, ū), where x = e 2t, 1 + e 4 )et e 4 e t), ū =, 1 2 e t+1)

17 Mordukhovich Subgradients of the Value Function in a Parametric Optimal Control Problem 167 is a solution of the problem corresponding to ᾱ, θ). ii) If α,θ ) V ᾱ, θ), thenα =, e) and θ =, e 1 t ). Indeed, for ᾱ, θ) = 1, 1),, ) ) the problem becomes J x, u) = x 2 1) + u u 2 + u 2 2) dt inf 1 ẋ 1 =2x 1 + u 1 ẋ 2 = x 2 + u 2 24) subject to x 1 ) = 1 x 2 ) = 1. By a direct computation, we see that the pair x, ū) satisfies 24). Besides, J x, ū) = e2 8 e Note that x, ū) is a solution of the problem. In fact, for all x, u) satisfying 24) we have J x, u) = x 2 1) + u u 2 + u 2 2) dt 1 ) x 2 1) + 1+u 2 2 dt 25) = x 2 1) + 1+ẋ2 x 2 ) 2) dt = ẋ2 x 2 ) 2 ) ẋ 2 dt. We now consider the variational problem 26) Ĵx 2 ):= ẋ2 x 2 ) 2 ẋ 2 ) dt inf. By solving the Euler equation with noting that Ĵ is a convex function, we obtain that ˆx 2 t) =ce t +1 c)e t is a solution of 26), where c is determined by c = ae 1 e 2 1 and a = x 2 1). Hence 27) Ĵx 2 ) Ĵˆx 2)=1+2 e2 1 e 2 c 1) c 1) ec 1) e. e Combining 25) with 27) and putting r = c 1, weobtain

18 168 Nguyen Thi Toan J x, u) 1+2 e2 1 e 2 r e r er e e2 8 e = J x, ū). Hence, x, ū) is a solution of the problem corresponding to ᾱ, θ). Assertion i) is proved. It remains to prove ii). We first prove that the solution map S is V -inner semicontinuous at w, z) with w =ᾱ, θ) and z = x, ū). Indeed, for α =α 1,α 2 ) R 2 and θ =θ 1,θ 2 ) L 2 [, 1], R 2 ), we recall the problem Let Jx, u, θ) = x 2 1) + u u 2 + u θ ) θ2 dt inf 1 ẋ 1 =2x 1 + u 1 + θ 1 ẋ 2 = x 2 + u 2 + θ 2 subject to x 1 ) = α 1 x 2 ) = α 2. L = λ u u 2 + u θ2 1 + ) θ2 2 1 ) ) + p 1 ẋ1 2x 1 u 1 θ 1 + p2 ẋ2 x 2 u 2 θ 2 and l x),x1) ) = λ x2 1) ) ) ) + λ 1 x1 ) α 1 + λ2 x2 ) α 2 be the Lagrangian and endpoint functional, respectively. By the Pontryagin maximum principle see [1] and [5]), there exist Lagrange multipliers λ >,λ 1,λ 2 and absolutely continuous functions p 1,p 2, not all zero, such that d 28) dt L ṗ 1 = 2p 1 ẋ = L x ṗ 2 = p 2, 29) and Lẋ) = l x) Lẋ1) = l x1) 3) L u = p 1 ) = λ 1 p 2 ) = λ 2 and 2λ u1 u 1 1+u 2 1 )2 ) p1 = 2λ u 2 p 2 =. By a simple computation, from 28)-3) we obtain ut) =u 1 t),u 2 t)) =, 1 2 e1 t). p 1 1) = p 2 1) = λ

19 Mordukhovich Subgradients of the Value Function in a Parametric Optimal Control Problem 169 From the equations ẋ 2 t) =x 2 t)+ 1 2 e1 t + θ 2 t), a.e.t [, 1] and x 2 t) = x 2 t)+ 1 2 e1 t,a.e.t [, 1], we have ẋ 2 t) x 2 t) = x 2 t) x 2 t) ) + θ 2 t), a.e.t [, 1]. It follows that 31) ẋ 2 t) x 2 t) x 2 t) x 2 t) + θ 2 t), a.e.t [, 1]. Since x 2 t) x 2 t) =α 2 1)+ t ẋ2 s) x 2 s) ) ds, we get from the above inequality that x 2 t) x 2 t) α α α t t t x 2 s) x 2 s) ds + x 2 s) x 2 s) ds + t x 2 s) x 2 s) ds + θ 2. By the Gronwall inequality see [5, Lemma 18.1.i]), we get ) x 2 t) x 2 t) α θ 2 e. θ 2 s) ds θ 2 s) ds Combining this with 31), we obtain ) ẋ 2 t) x 2 t) e α θ θ 2 t), a.e.t [, 1]. We note that x 2 x 2 1,2 = α ẋ 2 x 2 2 and ) ẋ 2 x 2 2 = ẋ 2 t) x 2 t) dt ) e α θ θ 2 2. So Hence, if ) x 2 x 2 1,2 e +1) α θ 2 2. α k 1,α k 2), θ k 1,θ k 2) ) V 1, 1),, ) )

20 17 Nguyen Thi Toan and x k 1,x k 2), u k 1,u k 2) ) S α k 1,α k 2), θ k 1,θ k 2) ), k N then x k 2 x 2. By using similar arguments, we also show that x k 1 x 1. It is easy to see that u k 1,uk 2 ) ū 1, ū 2 ). Thus, the solution map S is V -inner semicontinuous at ᾱ, θ), x, ū) ). From 23), we have ) 2 A =, B = 1 ) 1, T = 1 ) 1. 1 It is easy to see that conditions H 2 ) and H 3 ) are fulfilled. Since Lt, x, u, θ)=u u 2 + u θ1 2 + θ2, 2 1 we have Lt, x, u, θ) u 2 + θ 2 +1, L u t, x, u, θ) 2 u +1); L θ t, x, u, θ) 2 θ for all x, u, θ) R 2 R 2 R 2 and a.e. t [, 1]. Thus, condition H 1 ) is also valid. Hence, all conditions of Theorem 1.1 are fulfilled. Let α,θ ) V ᾱ, θ). By Theorem 1.1, there exists y =y 1,y 2 ) W 1,2 [, 1], R 2 ) such that ẏ 1 = 2y 1 ẏ + A T y = x Lt, x, ū, θ) ẏ 2 = y 2 y1) = g x1)). y 1 1) = y 2 1) = 1. This implies that y 1,y 2 )=ȳ 1, ȳ 2 )=,e 1 t ).By9),wehave θ = T T y + θ Lt, x, ū, θ) = T T y. It follows that θ t) =, e 1 t ). On the other hand, from 7) we get 32) α = g x1) ) + ) 1 L x t, xt), ūt), θt) dt A T t)yt)dt. Substituting y =ȳ into 32), we obtain α =, e). REFERENCES 1. V. M. Alekseev, V. M. Tikhomirov and S. V. Fomin, Optimal Control, Consultants Bureau, New York and London, 1987.

21 Mordukhovich Subgradients of the Value Function in a Parametric Optimal Control Problem J.-P. Aubin and I. Ekeland, Applied Nonlinear Analysis, Wiley, New York, J. M. Borwein and Q. J. Zhu, Techniques of Variational Analysis, Springer, New York, A. Cernea and H. Frankowska, A connection between the maximum principle and dynamic programming for constrained control problems, SIAM Journal on Control and Optimization, 44 25), L. Cesari, Optimization-Theory and Applications, Springer, New York, N. H. Chieu, B. T. Kien and N. T. Toan, Further results on subgradients of the value function to a parametric optimal control problem, Journal of Optimization Theory and Applications, DOI 1.17/s , N. H. Chieu and J.-C. Yao, Subgradients of the optimal value function in a parametric discrete optimal control problem, Journal of Industrial and Management Optimization, 6 21), F. H. Clarke, Optimization and Nonsmooth Analysis, SIAM, Philadelphia, F. H. Clarke, Methods of Dynamic and Nonsmooth Optimization, SIAM, Philadelphia, A. D. Ioffe, Euler-Lagrange and Hamiltonian Formalisms in dynamic optimization, Transactions of the American Mathematical Society, ), A.D.IoffeandV.M.Tihomirov,Theory of Extremal Problems, North-Holland Publishing Company, North-Holland, B. T. Kien, Y. C. Liou, N.-C. Wong and J.-C. Yao, Subgradients of value functions in parametric dynamic programming, European Journal of Operational Research, ), B. S. Mordukhovich, Variational Analysis and Generalized Differentiation I, Basis Theory, Springer, Berlin, B. S. Mordukhovich, Variational Analysis and Generalized Differentiation II, Applications, Springer, Berlin, B. S. Mordukhovich and N. M. Nam, Variational stability and marginal functions via generalized differentiation, Mathematics of Opeations Research, 3 25), B. S. Mordukhovich, N. M. Nam and N. D. Yen, Fréchet subdifferential calculus and optimality conditions in nondifferetiable programming, Optimization, 55 26), B. S. Mordukhovich, N. M. Nam and N. D. Yen, Subgradients of marginal functions in parametric mathematical programming, Mathematical Programming, ), M. Moussaoui and A. Seeger, Sensitivity analysis of optimal value functions of convex parametric programs with possibly empty solution sets, SIAM Journal on Optimization, ),

22 172 Nguyen Thi Toan 19. M. Moussaoui and A. Seeger, Epsilon-maximum principle of pontryagin type and perturbation analysis of convex optimal control problems, SIAM Journal on Control and Optimization, ), J.-P. Penot, Differetiability properties of optimal value functions, Canadian Journal of Mathematics, 56 24), R. T. Rockafellar and P. R. Wolenski, Convexity in Hamilton-Jacobi theory, I: dynamics and duality, SIAM Journal on Control and Optimization, 39 2), R. T. Rockafellar and P. R. Wolenski, Convexity in Hamilton-Jacobi theory II: envelope representation, SIAM Journal on Control and Optimization, 39 2), R. T. Rockafellar, Hamilton-Jacobi theory and parametric analysis in fully convex problems of optimal control, Journal of Global Optimization, ), A. Seeger, Subgradient of optimal-value function in dynamic programming: the case of convex system without optimal paths, Mathematics of Operations Research, ), N. T. Toan and B. T. Kien, Subgradients of the value function to a parametric optimal control problem, Set-Valued and Variational Analysis, 18 21), N. T. Toan and J.-C. Yao, Mordukhovich subgradients of the value function to a parametric discrete optimal control problem, Journal of Global Optimization, ), R. B. Vinter, Optimal Control, Birkhäuser, Boston, R. B. Vinter and H. Zheng, Necessary conditions for optimal control problems with state constraints, Transactions of the American Mathematical Society, ), Nguyen Thi Toan School of Applied Mathematics and Informatics Hanoi University of Science and Technology 1 Dai Co Viet, Hanoi Vietnam toandhv@gmail.com