arxiv: v1 [math.oc] 27 Jan 2016

Transcription

1 Variational Analysis of the Ky Fan k-norm Chao Ding arxiv: v1 [math.oc] 7 Jan 016 October 19, 015 Abstract In this paper, we will study some variational properties of the Ky Fan k-norm θ = k of matrices, which are closed related to a class of basic nonlinear optimization problems involving the Ky Fan k-norm. In particular, for the basic nonlinear optimization problems, we will introduce the concept of nondegeneracy, strict complementarity and the critical cones associated with the generalized equations. Finally, we present the explicit formulas of the conjugate function of the parabolic second order directional derivative of θ, which will be referred to as the sigma term of the second order optimality conditions. The results obtain in this paper provide the necessary theoretical foundations for future work on sensitivity and stability analysis of the nonlinear optimization problems involving the Ky Fan k-norm. Key Words: Ky Fan k-norm, nondegeneracy, critical cone, second order tangent sets AMS subject classifications: 65K10, 90C5, 90C33 1 Introduction Let IR m n bethe vector space of all m n real matrices equippedwith the inner product Y,Z := TrY T Z for Y andz in IR m n, where Tr denotes the trace, i.e., the sumof the diagonal entries, of a squared matrix. For simplicity, we always assume that m n. For any given positive integer 1 k m, denote θ := k the matrix Ky Fan k-norm, i.e., the sum of k largest singular values of matrices. In particular, 1 coincides with the spectral norm of the matrices, i.e., the largest singular value of matrices; m is the nuclear norm of matrices, i.e., the sum of singular values of matrices. It is well-known that ϑz := Z k = max{ Z, Z /k} for Z IR m n is the dual norm of k cf. [1, Exercise IV.1.18]. Since θ is a matrix norm convex, closed, positively homogeneous and θ0 = 0, we obtain from [33, Theorem 13.5 & 13.] that the conjugate function θ = δ θ0 is just the indicator function of the subdifferential θ0 of θ at 0. Moreover, it can be verified directly from the definition of dual norm that θ0 coincides with the unit ball under the dual norm ϑ, i.e., θ0 = B k := {S IR m n ϑs 1}. Considerthefollowing nonlinear optimization probleminvolving theky Fan k-normθ = k min{fxθgx x X}, 1 Institute of Applied Mathematics, Chinese Academy of Sciences, Beijing, P.R. China. dingchao@amss.ac.cn. This work is supported in part by the National Natural Science Foundation of China Grant No

2 where X is a finite dimensional real vector space equipped with a scalar product,, f : X IR is a continuously differentiable real value function, and g : X IR m n is a continuously differentiable function. Since θ is convex and finite everywhere, it is well-known [34, Example 10.8] that for a locally optimal solution x X of 1, there always exists a Lagrange multiplier S IR m n, together with x satisfying the following first order optimality condition, namely the Karush-Kuhn-Tucker KKT condition: f xg x S = 0 and S θx, where X := g x, f x X is the gradient of f at x, g x : IR m n X is the adjoint of the derivative mapping g x. Note that if X = IR m n and gx := x is the identity mapping, then the KKT condition becomes the following generalized equation: 0 fx θx. Note also that if the function θ is replaced by the indicator function δ K of a set K in a finite dimensional real vector space, then the nonlinear optimization problem 1 becomes min fx s.t. gx K. 3 During the last three decades, considerable progress has been made in the variational analysis related to the problem 3 [34, 1,, 19, 4]. In particular, for the general non-polyhedral set K e.g., the second-order cone and the positively semidefinite SDP matrices cone, by employing the well studied properties of the variational inequality S N K x, some important properties of 3, such as the constraint nondegeneracy, second order optimality conditions, strong regularity, full stability and calmness, are studied recently by various researchers [, 35, 5]. In order to extend those results to the optimization problems involving the Ky Fan k-norm, we need first study the variational properties of 1, especially the properties of the generalized equation S θx and its equivalent dual problem X θ S. Although the optimization problem 1 seems extremely simple, many fundamental and important issues such as the concept of nondegeneracy, the characterizations of critical cones and the second order optimality conditions, are not studied yet in literature. The main purpose of this paper is to build up the necessary variational foundations for the future work on the nonlinear optimization problems involving the Ky Fan k-norm. Certainly, instead of the basic model 1, one can consider its various modifications, e.g., the nonlinear optimization problems involving the Ky Fan k-norm with equality and conic constraints. In particular, the following convex composite matrix optimization problems involving the Ky Fan k-norm frequently arise in various applications such as the matrix norm approximation, matrix completion, rank minimization, graph theory, machine learning, etc [15, 36, 37, 9, 4, 5, 6, 7, 40, 8, 3, 14,, 11, 17, 3]: min 1 X,Y,QX,Y C,X,Y θx s.t. AX,Y = b, Y K, 4 where Y is a finite dimensional real vector space, Q : IR m n Y IR m n Y is a positively semidefinite self-adjoint linear operator, A : IR m n Y IR p is a linear operator, C IR m n Y

3 and b IR p are given data, and K Y is a closed convex cone e.g., the positive orthant, secondorder cone of vectors, positive semidefinite matrices cone. As the initial step, in this paper, we will mainly focus on the fundamental model 1, since the obtained variational results will provide the necessary theoretical foundations for the study of more complicate model, e.g., 4. More precisely, we will study the concepts of nondegeneracy and strict complementary to locally optimal solutions of 1. Also, we will define and provide the complete characterizations of the critical cones associated with the generalized equation S θx and its dual problem X θ S. Another important variational property studied in this paper is the conjugate function of the parabolic second order directional derivative of the Ky Fan k-norm θ, which equals to the support function of the second order tangent set of the epigraph of θ. This conjugate function is closely related to the second order optimality conditions of the problem 1. Note that the epigraph of θ is not polyhedral. In general, the conjugate function of the parabolic second order directional derivative of the Ky Fan k-norm θ will not vanish in the corresponding second order optimality conditions, and will be referred to as the sigma term, provides the second order information of θ. In this paper, we provide the explicit expression of this sigma term. Consequently, it becomes possible to establish the second order optimality conditions of the problem 1 and study many corresponding sensitivity properties, e.g., the second order optimality conditions and the characterization of strong regularity of the KKT solutions. The remaining parts of this paper are organized as follows. In Section, we introduce some preliminary results on the differential properties of eigenvalue values and vectors of symmetric matrices and singular values and vectors of matrices. In Section 3, we study the properties of the solution of the GE S θx, which arises from the KKT condition and its equivalent dual form X θ S. We introduce the nondegeneracy and strict complementarity of 1 in Section 4. In Section 5, we introduce and study the critical cones associated with the GE S θx and X θ S. The second order properties of the Ky Fan k-norm θ are studied in Section 6. We conclude our paper in the final section. Below are some common notations to be used: For any Z IR m n, we denote by Z ij the i,j-th entry of Z. For any Z IR m n, we use z j to represent the jth column of Z, j = 1,...,n. Let J {1,...,n} be an index set. We use Z J to denote the sub-matrix of Z obtained by removing all the columns of Z not in J. So for each j, we have Z {j} = z j. Let I {1,...,m} and J {1,...,n} be two index sets. For any Z IR m n, we use Z IJ to denote the I J sub-matrix of Z obtained by removing all the rows of Z not in I and all the columns of Z not in J. We use to denote the Hardamard product between matrices, i.e., for any two matrices X and Y in IR m n the i,j-th entry of Z := X Y IR m n is Z ij = X ij Y ij. Preliminaries In this section, we list some useful preliminary results on the eigenvalues of symmetric matrices and the singular values of matrices, which are useful for our subsequent analysis. Let S n be the space of all real n n symmetric matrices and O n be the set of all n n orthogonal matrices. Let X S n be given. We use λ 1 X λ X... λ n X to denote 3

4 the real eigenvalues of X counting multiplicity being arranged in non-increasing order. Denote λx := λ 1 X,λ X,...,λ n X T IR n and ΛX := diagλx, where for any x IR n, diagx denotes the diagonal matrix whose i-th diagonal entry is x i, i = 1,...,n. Let P O n be such that X = PΛXP T. 5 We denote the set of such matrices P in the eigenvalue decomposition 5 by O n X. Let ω 1 X > ω X >... > ω r X be the distinct eigenvalues of X. Define the index sets a k := {i λ i X = ω k X, 1 i n}, k = 1,...,r. 6 For each i {1,...,n}, we define l i X to be the number of eigenvalues that are equal to λ i X but are ranked before i including i and s i X to be the number of eigenvalues that are equal to λ i X but are ranked after i excluding i, respectively, i.e., we define l i X and s i X such that λ 1 X... λ i li XX > λ i li X1X =... = λ i X =... = λ isi XX > λ isi X1X... λ n X. 7 In later discussions, when the dependence of l i and s i, i = 1,...,n, on X can be seen clearly from the context, we often drop X from these notations. Next, we list some useful results about the symmetric matrices which are needed in subsequent discussions. The inequality in the following lemma is known as Fan s inequality [13]. Lemma.1 Let Y and Z be two matrices in S n. Then Y,Z λy T λz. 8 where the equality holds if and only if Y and Z admit a simultaneous ordered eigenvalue decomposition, i.e., there exists an orthogonal matrix U O n such that Y = UΛYU T and Z = UΛZU T. The following proposition on the directional differentiability of the eigenvalue function λ is well known. For example, see [0, Theorem 7] and [38, Proposition 1.4]. Proposition.1 Let X S n have the eigenvalue decomposition 5. Then, for any S n H 0, we have λ i X H λ i X λ li P T a k HP ak = O H, i α k, k = 1,...,r, 9 where for each i {1,...,n}, l i is defined in 7. Hence, for any given direction H S n, the eigenvalue function λ i is directionally differentiable at X with λ i X;H = λ l i P T a k HP ak, i a k, k = 1,...,r. Letk {1,...,r}befixed. ForthesymmetricmatrixP T a k HP ak S a k, considertheeigenvalue decomposition P T a k HP ak = RΛP T a k HP ak R T, 10 where R O a k. Denote the distinct eigenvalues of P T a k HP ak by µ 1 > µ >... > µ r. Define ã j := {i λ i P T a k HP ak = µ j,1 i a k }, j = 1,..., r. 11 4

5 For each i a k, let l i {1,..., a k } and k {1,..., r} be such that li := l li P T a k HP ak and l i ã k, 1 where l i is defined by 7. Let X and X be two finite dimensional real Euclidean spaces. We say that a function Φ : X X is parabolic second order directionally differentiable at x X, if Φ is directionally differentiable at x and for any h,w X, Φxth 1 lim t w Φx tφx;h t 0 1 exists; t and the above limit is said to be the parabolic second order directional derivative of Φ at x along the directions h and w, denoted by Φ x;h,w. The following proposition [38, Proposition.], provides the explicit formula of the parabolic second order directional derivative of the eigenvalue function. Proposition. Let X S n have the eigenvaluedecomposition 5. Then, for any givenh,w S n, we have for each k {1,...,r} ] λ i X;H,W = λ li Rã kp T a T k [W HX λ i I n H P ak, i a Rã k k, 13 where Z IR n n is the Moore-Penrose pseudoinverse of the square matrix Z IR n n. Let X IR m n be given. Without loss of generality, assume that m n. We use σ 1 X σ X... σ m X to denote the singular values of X counting multiplicity being arranged in non-increasing order. Define σx := σ 1 X,σ X,...,σ m X T and ΣX := diagσx. Let X IR m n admit the following singular value decomposition SVD: X = U [ΣX 0]V T = U [ΣX 0][V 1 V ] T = UΣXV T 1, 14 where U O m and V = [V 1 V ] O n with V 1 IR n m and V IR n n m. The set of such matrices pair U,V in the SVD 14 is denoted by O m,n X, i.e., O m,n X := { U,V O m O n X = U [ΣX 0]V T}. Define the three index sets a, b and c by a := {i σ i X > 0, 1 i m}, b := {i σ i X = 0, 1 i m} and c := {m1,...,n}. 15 Let ν 1 X > ν X >... > ν r X > 0 be the distinct nonzero singular values of X. Without causing any ambiguity, we also use a k to denote the following index sets a k := {i σ i X = ν k X, 1 i m}, k = 1,...,r. 16 For the sake of convenience, let a r1 := b. For each i {1,...,m}, we also define l i X to be the number of singular values that are equal to σ i X but are ranked before i including i and s i X 5

6 to be the number of singular values that are equal to σ i X but are ranked after i excluding i, respectively, i.e., we define l i X and s i X such that σ 1 X... σ i li XX > σ i li X1X =... = σ i X =... = σ isi XX > σ isi X1X... σ m X. 17 In later discussions, when the dependence of l i and s i, i = 1,...,m, on X can be seen clearly from the context, we often drop X from these notations. The inequality in the following lemma is known as von Neumann s trace inequality [7]. Lemma. Let Y and Z be two matrices in IR m n. Then Y,Z σy T σz, 18 where the equality holds if Y and Z admit a simultaneous ordered singular value decomposition, i.e., there exist orthogonal matrices U O m and V O n such that Y = U[ΣY 0]V T and Z = U[ΣZ 0]V T. by For notational convenience, define two linear operators S : IR p p S p and T : IR p p IR p p SZ := 1 Z ZT and TZ := 1 Z ZT Z IR p p. 19 The following proposition on the directional derivatives of the singular value functions can be obtained directly from Proposition.1. For more details, see [1, Section 5.1]. Proposition.3 Let X IR m n have the singular value decomposition 14. For any IR m n H 0, we have σ i X H σ i X σ i X;H = O H, i = 1,...,m, 0 with σ ix;h = { λli SU T ak HV ak if i a k, k = 1,...,r, σ li [U T b HV b U T b HV ] if i b, 1 where for each i {1,...,m}, l i is defined in 17. Similarly, one can derive the following explicit formulas of the parabolic second order directional derivatives of the singular value functions from Proposition., directly. For more details, see [4, Theorem 3.1]. Proposition.4 Let X IR m n have the singular value decomposition 14. Suppose that the direction H,W IR m n are given. i If σ i X > 0, then σ i X;H,W = R λ li T α k SU T ak WV ak Ω ak X,H R α k, 6

7 where k {1,...,r} such that i a k, Ω ak X,H S m is given by Ω ak X,H = SU T HV 1 ak T ΣX ν k XI m SU T HV 1 ak TU T HV 1 ak T ΣX ν k XI m TU T HV 1 ak 1 ν k X UT a k HV V T H T U ak, the matrix R O a k satisfies SU T a k HV ak = RΛSU T a k HV ak R T, and { α j } r j=1 and l i, k be defined by 11 and 1 respectively for SU T a k HV ak. ii If σ i X = 0 and σ li [U T b HV b U T b HV ] > 0, then σ i X;H,W = λ li SE ã k[u T b T ZV b Ub T ZV]Fã k, where Z = W HX H IR m n, X IR n m is the Moore-Penrose pseudoinverse of X IR m n, E O b, F = [F 1 F ] O b n m satisfy [U T b HV b U T b HV ] = E[Σ[U T b HV b U T b HV ] 0]F T, l i {1,..., ã k } and k {1,..., r} such that l i = l li SEã k[u T b TZV b Ub TZV ]Fã k and l i ã k, ã j, j = 1,..., r are the index sets of [Ub THV b Ub THV ] defined by ã j := {i σ i [U T b HV b U T b HV ] = ν j, 1 i b }, and ν 1 > ν >... > ν r are the nonzero distinct singular values of [U T b HV b U T b HV ]. iii If σ i X = 0 and σ li [Ub THV b Ub THV ] = 0, then σ ix;h,w = E T b [U T σ li b ZV b Ub T ZV ][F b F ], where Z = W HX H IR m n, b := {i σ i [Ub THV b Ub THV ] = 0, 1 i b } and l i = l li E T b [Ub TZV b Ub TZV ][F b F ] is defined by 17 with respect to E T b [Ub TZV b Ub TZV ][F b F ]. 3 The generalized equations In this section, we first study some properties of the following simple generalized equation GE which is equivalent to the following dual form 0 S θx, 3 0 X θ S. 4 Since θ = δ Bk, it follows from [33, Theorem 3.5] that 3 and 4 are also equivalent to the following complementarity problem X,θX K, S, 1 K and X,θX,S, 1 = 0, 5 7

8 where K is the epigraph of θ = k, i.e., K = epiθ = { X,t IR m n IR t X k } 6 and K is the polar cone of K given by K = ρ 0ρ θ0, 1 = epiϑ with ϑ = k. On the other hand, it is well-known [6] see also [33, Theorem 31.5] that X,S is a solution of the GE 3 or 4 if and only if X Pr θ X S = 0 S Pr θ X S = 0, where Pr θ : IR m n IR m n is the Moreau-Yosida proximal mapping of θ, and Pr θ : IR m n IR m n is the Moreau-Yosida proximal mapping of θ. Denote X := X S. Let X admit the following singular value decomposition X = U [ΣX 0]V T. 7 Let σ = σx, σ = σx and u = σs be the singular values of X, X and S, respectively. Since and k are unitarily invariant, we know from von Neumann s trace inequality Lemma. that X = U [Diagσ 0]V T and S = U [Diagu 0]V T with σ = gσ and u = σ gσ, 8 where g : IR m IR m is the Moreau-Yosida proximal mapping of the vector k-norm i.e., the sum of thek largest components inabsolutevalue of any vector inir m. Thepropertiesof theproximal mapping g have been studied recently in [41], e.g., for any given x IR m, the unique optimal solution gx IR m can be computed within Om arithmetic operations see [41, Section 3.1] for details. The following simple observations are useful for our subsequence analysis, which can be obtained directly from the characterization of the subdifferential of θ = k cf. [39, 8]. Lemma 3.1 σ and u are the singular values of the solution X,S of the GE 3 or 4 if and only if σ and u satisfy the following conditions. i If σ k > 0, then u α = e α, 0 u β e β, u i = 0 and u γ = 0, 9 i β where 0 k 0 k 1 and k k 1 m are two integers such that σ 1... σ k0 > σ k0 1 =... = σ k =... = σ k1 > σ k σ m 0 30 and α = {1,...,k 0 }, β = {k 0 1,...,k 1 } and γ = {k 1 1,...,m}. 31 8

9 ii If σ k = 0, then u α = e α, 0 u β e β where 0 k 0 k 1 is the integer such that and u i 0, 3 i β σ 1 σ k0 > σ k0 1 =... = σ k =... = σ m = 0 33 and α = {1,...,k 0 } and β = {k 0 1,...,m}. 34 For notational convenience, we use β 1, β and β 3 to denote the index sets β 1 := {i β u i = 1}, β := {i β 0 < u i < 1} and β 3 := {i β u i = 0}. 35 For X = X S, let a, b and c be the index sets defined by 15. We use a 1,...,a r to denote the index sets defined by 16 with respect to X and a r1 = b for the sake of convenience. Thus, by Lemma 3.1, we know that if σ k > 0, then there exist integers r 1 {0,1,...,r 1}, 1 and r 1 1 r 1 r 1 such that α = a l, β 1 = l= 1 a l, β = r 1 l= 1 a l, β 3 = r 1 l= r 1 1 a l and γ = r1 l=r 1 1 if σ k = 0, then there exist integers {0,1,...,r 1} and 1 such that α = a l, β 1 = l= 1 a l, β = r l= 1 a l ; 36 a l and β 3 = b. 37 Moreover, we know that for each l {1,..., }, σ i = σ j for any i,j a l, which implies that we can use ν 1 >... > ν r0 > 0 to denote those common values. Similarly, if σ k > 0, we use µ r0 1 >... > µ r1 > 0 to denote the corresponding common values of u; if σ k = 0, we use µ r0 1 >... > µ r > 0 to denote the corresponding common values of u. 4 The nondegeneracy and strict complementarity In this section, we shall introduce the nondegeneracy and strict complementarity of the optimization problem 1. To do so, let us consider the following conic reformulation of 1: min fxt s.t. gx,t K, 38 where K = epiθ. Let x, t be a feasible point of 38. Denote X = g x IR m n. Recall the definition [34, Definition 6.1] of the tangent cone T K X, t of K at the given point X, t K, i.e., T K X, t = { H,τ IR m n IR ρ n 0, dist X, tρ n H,τ,K = oρ n }. 9

10 For any convex function φ : IR m n,, we know from [9, Theorem.4.9] that { } T epiφ Y,φY = epiφ Y; := H,τ IR m n IR φ Y;H τ, Y IR m n. 39 Therefore, for θ = k, we know from Proposition.3 that T K X,θX = { T 0 H,τ tru αhv α λ i SU T βhv β { H,τ tru T αhv α 0 τ } if σ k X > 0, [ ] σ i U T βhv β U T βhv τ } if σ k X = Define G : X IR IR m n IR by Gx,t := gx,t, x,t X IR. Robinson s CQ [30] for 38 at a given feasible point x, t can be written as G x, tx IRT K X, t = IR m n IR. 41 Proposition 4.1 For any x X, Robinson s CQ 41 for 38 holds at x,θg x. Proof. Note that the directional derivative θ X; of the Ky Fan k-norm is finite everywhere. Therefore, the results can be derived directly from 41 and 39. In fact, we only need to show that for any given X,t IR m n IR, there exists h,η X IR and H,τ T K X, t with t = θg x such that g xh,ηh,τ = X,t. Let H = X and τ = θ X;X. By choosing h = 0 and η = t τ, we know that the above equality holds trivially. As we mentioned in Section 1, for a locally optimal solution x to the optimization problem 1, the corresponding Lagrange multiplier always exists. In next proposition, we show that the set of Lagrange multipliers of 1 is also convex, bounded and compact. Proposition 4. Let x X be a locally optimal solution to the problem 1. The set of Lagrange multipliers of 1 is a nonempty, convex, bounded and compact subset of IR m n. Proof. It is easy to see that x X is a locally optimal solution of 1 if and only if x,θg x is a locally optimal solution of 38. Moreover, by 5, we know that there exists a Lagrange multiplier S IR m n if and only if there exists S IR m n such that the following KKT condition of 38 holds at x,θg x,s, 1: fxg x S = 0, ξ 1 = 0, 4 gx,t,s,ξ = 0, gx,t K and S,ξ K. On the other hand, it is well-known [43] that for a locally optimal solution of 38, the corresponding set of Lagrange multipliers is nonempty, convex, bounded and compact if and only if Robinson s CQ holds. Therefore, the result follows from Proposition 4.1 directly. 10

11 Next, let us study the concept of nondegeneracy for the optimization problem 1. For any convex function φ : IR m n, and Y IR m n, the lineality space of T epiφ Y,φY, i.e., the largest linear subspace in T epiφ Y,φY, can be written as = = lint epiφ Y,φY = T epiφ Y,φY T epiφ Y,φY { } H,τ IR m n IR φ Y;H τ φ Y; H { } H,τ IR m n IR φ Y;H = φ Y; H = τ. 43 The last equation of 43 follows from [33, Theorem 3.1], directly. For the Ky Fan k-norm θ = k, define the linear subspace T lin X IR m n by { } T lin X := H IR m n θ X;H = θ X; H. 44 If S θx, then, by Proposition.3, we have { } H IR m n SU T T lin β X = HV β = τi β for some τ IR { H IR m n [ U T β HV β U T β HV ] } = 0 if σ k X > 0, if σ k X = 0, 45 where U O m and V O n are eigenvectors of X = XS, and the index set β is defined in 31 if σ k X > 0 and in 34 if σ k X = 0. For the problem 38, the concept of Robinson s constraint nondegeneracy [31, 3] can be specified as follows. The constraint nondegeneracy for 38 holds at the feasible point x, t if G x, tx IRlin T K X, t = IR m n IR, 46 where the lineality space lin T K X, t is given by 43 with respect to θ = k. Proposition 4.3 The constraint nondegeneracy 46 for 38 holds at x, θx if and only if g xx T lin X = IR m n, 47 where T lin X IR m n is the linear subspace defined by 44. Therefore, we say that the nondegeratacy for the problem 1 holds at x if 47 holds. Proof. For any given X IR m n, by 46, we know that there exists h X, H,η lin T K X, t such that g xh, ηh,η = X,0. Since H T lin X, we know that 47 holds. Conversely, for any X,t IR m n IR, by 47, we know that there exists h X and H T lin X such that g xhh = X. Denote τ = θ X;H. By taking η = t τ, we obtain that g xh,ηh,τ = X,t, 11

12 which implies that the constraint nondegeneracy 46 holds at x, θx. Let x X be a locally optimal solution of 1. Denote X = g x. Let β be the index set defined in 31 if σ k X > 0 and in 34 if σ k X = 0. The following definition of the strict complementarity of 1 can be regarded as a generalization of the strict complementarity for the constraint optimization problem cf. [, Definition 4.74]. Definition 4.1 We say the strict complementarity condition holds at x X if there exists S ri θx such that f xg x S = By Lemma 3.1, one can derive the following proposition easily. For simplicity, we omit the detail proof here. Proposition 4.4 The strict complementarity condition holds at x X if and only if there exists S θx such that 48 holds and i if σ k X > 0, then 0 < σ β S < e β ; ii if σ k X = 0, then σ β S < e β and i β σ is < k k 0, Proposition 4.5 Let x X be a locally optimal solution of 1. Denote X = g x. If x is nondegenerate, then S satisfying is unique. Conversely, if S satisfying is unique and the strict complementarity condition holds at x, then x is nondegenerate. Proof. The following proof is a slight modification of the proof of [, Proposition 4.75]. Suppose that x is nondegenerate and let S and S satisfy. Then, we know that g x S S = 0, which implies that := S S [g xx]. Denote X = X S and X = X S. Suppose that X and X admit the SVD: X = U[ΣX 0]V T and X = U [ΣX 0]V T, whereu,u O m andv,v O n. By 8, weknow that both U,VandU,V are eigenvalue vectors of X. Therefore, it follows from [10, Proposition 5] that if σ k X > 0, then there exist orthogonal matrices Q 1 O α, Q O β, Q 3 O γ and Q 3 O γ n m such that U = U Q Q Q 3 and V = V Q Q Q 3 if σ k X = 0, then there exist orthogonal matrices Q 1 O α, Q O β and Q O β n m such that [ ] [ ] U Q1 0 = U and V Q1 0 = V 0 Q 0 Q. Therefore, by 8, we know from Lemma 3.1 that if σ k X > 0, then U T HV = 0 SU T β HV β 0 0 with trsu T β HV β = ;

13 if σ k X = 0, then [ U T HV = 0 U T βhv β U T βhv Thus, we know from 45 that in both cases, ]. 50,H = U T V,U T HV = 0 H T lin X, which implies that [ T lin X ]. Therefore, by 47, we know that = 0, i.e., S satisfying is unique. Conversely, since the strict complementarity condition holds at x, we know that the unique Lagrange multiplier S θx satisfying i and ii of Proposition 4.4. Let X = X S admit the SVD 7. Suppose that the constraint nondegenerate condition 47 does not hold at X, i.e., there exists 0 H [g xx] [ T lin X ]. Therefore, we know that g X H = 0. Moreover, by 45, we know that if σ k X > 0, then 49 holds; if σ k X = 0, then 50 holds. Since g X H = 0, we know that for any ρ, f xg X S ρh = 0. Moreover, since S satisfies i and ii of Proposition 4.4, by 49 and 50, we know from Lemma 3.1 that for ρ > 0 small enough, S ρh θx. This contradicts the uniqueness of S. Remark 4.1 Let X θ S. For the dual norm ϑ = k, since S, 1 K, we have T K S, 1 = We define the linear subspace T lin S IR m n by { { H,τ IR m n IR ϑ S;H τ } if ϑs = 1, IR m n IR if ϑs < 1. T lin S := { { H IR m n ϑ S;H = ϑ S; H = 0 } if ϑs = 1, IR m n if ϑs < For the case that ϑs = max{ S, S /k} = 1, we know from Proposition.3 that if S < k, then T lin S = { H IR m n S[U α U β1 ] T H[V α V β1 ] = 0 } ; 5 if S = k, then } T {H lin S = IR m n SU T α β 1 HV α β1 = 0, tru T β HV β = 0, U T β 3 γ HV β 3 γ c = 0, 53 where U O m and V O n are eigenvectors of X = X S, the index set β is defined in 31 if σ k X > 0 and in 34 if σ k X = 0, and β 1, β and β 3 are the index sets defined by

14 5 The critical cones From now on, let us always assume that X = g x and S are solutions of the GEs 3 and 4. Therefore, the critical cones associated with the GEs 3 and 4 can be defined correspondingly from the critical cones associated the complementarity problem 5. Firstly, consider the GE 3. Denote X,t = X S,θX 1. The critical cone of K at X, t associated with the complementarity problem in 5, is defined as Thus, we know from 39 that CX,t;K = T K X,θX S, H,τ CX,t;K { H CX; θx, τ = S,H, 55 where CX; θx IR m n is defined by CX; θx := { H IR m n θ X;H S,H }. 56 Since θ X; is a positively homogeneous convex function with θ X;0 = 0, CX; θx is indeed a closed convex cone. We call CX; θx the critical cone of θx at X = X S, associated with the GE 3. Next, we present the following proposition on the characterization of the critical cone CX; θx. Proposition 5.1 Suppose that X,S IR m n IR m n is a solution of the GE 3. Let X = X S admit the SVD 7. Then, which is equivalent to the following conditions. H CX; θx θ X;H = S,H, 57 i If σ k X > 0, then there exists some τ IR such that λ β1 SU T β 1 HV β1 τ λ 1 SU T β 3 HV β3 and SU T βhv β = SU T β 1 HV β τi β SU T β 3 HV β3. ii If σ k X = 0 and S = k, then there exists some τ 0 such that λ β1 SU T [U T β 1 HV β1 τ σ 1 b HV b U T b HV ] and ] [U T βhv β U T βhv = SU T β 1 HV β τi β U T b HV b U T b HV. 14

15 iii If σ k X = 0 and S < k, then SU T β 1 HV β1 0 and ] [U T βhv β U T βhv = SUT β 1 HV β Proof. Denote σ = σx and u = σs. By S,H = U T SV,U T HV = [Diagu 0],U T HV, we know from Lemma 3.1 that for any H IR m n, tru T α HV α Diagu β,su T β HV β S,H = ] tru T αhv α [Diagu β 0], [U T βhv β U T βhv if σ k > 0, if σ k = 0. Thus, by combining with Fan s inequality Lemma.1 and von Neumann s trace inequality Lemma., we obtain that for any H IR m n, if σ k > 0, and if σ k = 0, Diagu β,s H ββ u T β λs H 0 ββ λ i S H ββ, 58 [Diaguβ 0 ], [ Hββ Hβc ] u T β σ [ Hββ Hβc ] where H = U T HV. Therefore, we know from 40 that ] σ i [ Hββ Hβc, 59 H CX; θx θ X;H = S,H the equalities in 58 and 59 hold. Consider the following two cases. Case 1 σ k > 0. It follows from Lemma.1 that the first equality of 58 holds if and only if Diagu β and S H ββ admit a simultaneous ordered eigenvalue decomposition, i.e., there exists R O β such that 0 Diagu β = RDiagu β R T and S H ββ = RΛS H ββ R T. 60 Let r 1 {0,1,...,r 1}, 1 and r 1 1 r 1 r 1 be the integers such that 36 holds. Therefore, the orthogonal matrix R O β has the following block diagonal structure: R = R R R 3 with R = R , R r 1 15

16 where R 1 O β 1, R O β, R 3 O β 3 and R l O a l, l = 1,..., r 1. Thus, 60 holds if and only if S H ββ S β has the following block diagonal structure: S H β1 β S H 0 0 a r0 1a r0 1 S H ββ = S H a r1 0 a r S H β3 β 3 and the elements of λs H β1 β 1,λS H,...,λS H a r0 1a r0 1 a r1,λs H a r1 β3 β 3 are in nonincreasing order and are the eigenvalues of the symmetric matrix S H ββ. On the other hand, by 9, we know that u β1 = e β1, 0 < u β < e β, u β3 = 0 and e β,u β = 0. Then, we can verify that the second equality of 58 holds if and only if λ i S H ββ = λ j S H ββ i,j { β 1 1,..., β 1 β }. 6 In fact, it is clear that 6 implies the second equality of 58 holds. Conversely, without loss of generality, assume that β, then k k 0 { β 1 1,..., β 1 β }. Suppose that there exists i { β 1 1,..., β 1 β } but i k k 0 such that λ i S H ββ > λ 0 S H ββ or λ 0 S H ββ > λ i S H ββ. Then, since 0 < u β < e β and e β,u β = k k 0, for both cases, we always have, = 0 0 λ i S H ββ u T β λs H ββ λ i S H ββ 1 u β i β i= 0 1 u β i λ i S H ββ > 0 λ 0 S H ββ 1 u β i β i= = λ 0 S H ββ 0 u β i u β i λ 0 S H ββ β i= 0 1 u β i = 0, which implies that the second equality of 58 does not hold, which contradicts the assumption. Therefore, we know that H CX; θx if and only if i holds. Case σ k = 0. We know from Lemma. that the first equality of 59 holds if and only if [Diagu β 0] and [ Hββ Hβc ] admit a simultaneous ordered SVD, i.e., there exist orthogonal matrices E O β and F O β n m such that [Diagu β 0] = E[Diagu β 0]F T and [ Hββ Hβc ] = E[Σ[ Hββ Hβc ] 0]F T. 63 Let {0,1,...,r 1} and 1 be the integers such that 37 holds. Therefore, it follows from [10, Proposition 5] that there exist orthogonal matrices Q 1 O β 1, Q O β, 16

17 Q 3 Q β3 and Q 3 O β3 n m such that Q Q Q E = 0 Q 0 and F = 0 Q 0 with Q = , 0 0 Q Q Q r 64 whereq l O a r l 0, l = 1,...,r. Thus, 63 holds if and only if [ Hββ ] Hβc has the following block diagonal structure: H ar0 1a r Ha r0 1a r0 1 ] [ Hββ Hβc = Hara r Hbb Hbc with H al a l S al, l = 1,...,r, and the elements of h := λ H ar0 1a,λ H,...,λ H r0 1 a r0 1a r0 1 ara r,σ[ H bb Hbc ] IR m are nonnegative and in non-increasing order and h = σ [ H ββ Hβc ]. On the other hand, by 3, we know that u β1 = e β1, 0 < u β < e β, u β3 = 0 and e β,u β 0. Then, we may conclude that the second equality of 59 holds if and only if { σi [ Hββ Hβc ] = σ j [ Hββ Hβc ] i,j { β 1 1,..., β 1 β } if e β,u β = 0, σ i [ Hββ Hβc ] = 0 i { β 1 1,..., β } if e β,u β < 0, 65 In fact, it is evident that 65 implies that the second equality of 59 holds. Conversely, consider the following two sub-cases. Case.1 S = k, i.e., e β,u β = k k 0. Without loss of generality, assume that β, which implies 0 { β 1 1,..., β 1 β }. Suppose that there exists i { β 1 1,..., β 1 β } but i k k 0 such that σ i [ Hββ Hβc ] > σ 0 [ Hββ Hβc ] or σ 0 [ Hββ Hβc ] > σ i [ Hββ Hβc ]. Then, since 0 < u β < e β and e β,u β = 0, for both cases, we always have = 0 0 σ i [ Hββ Hβc ] u T β σ [ H ββ Hβc ] σ i [ Hββ Hβc ] 1 u β i β i= 0 1 u β i σ i [ Hββ Hβc ] > 0 σ 0 [ Hββ Hβc ] 1 u β i β i= 0 1 = σ 0 [ Hββ Hβc ] 0 0 u β i 17 u β i σ 0 [ Hββ Hβc ] β i= 0 1 u β i = 0,

18 which implies that the second equality of 59 does not hold, which contradicts the assumption. Therefore, we know that H CX; θx if and only if ii holds. Case. S < k, i.e., e β,u β < k k 0. We know that β β 3 and k k 0 { β 1 1,..., β }. Suppose that 65 does not hold. Then, we know that either there exists i { β 1 1,..., β } such that i < k k 0 and σ i [ Hββ Hβc ] > σ 0 [ Hββ Hβc ] = 0 or σ 0 [ Hββ Hβc ] > 0. For the case that σ i [ Hββ Hβc ] > σ 0 [ Hββ Hβc ] = 0, since 0 < u β < e β, we have = > σ i [ Hββ Hβc ] u T β σ [ H ββ Hβc ] σ i [ H ββ H βc ] 1 u β i σ 0 [ Hββ Hβc ] 1 u β i β i= 0 1 β i= 0 1 u β i σ i [ H ββ H βc ] u β i σ 0 [ Hββ Hβc ] = 0. For the case that σ 0 [ Hββ Hβc ] > 0, since e β,u β < k k 0, we obtain that = 0 0 σ i [ Hββ Hβc ] u T β σ [ H ββ Hβc ] σ i [ Hββ Hβc ] 1 u β i β i= 0 1 u β i σ i [ Hββ Hβc ] 0 σ 0 [ Hββ Hβc ] 1 u β i β i= 0 1 = σ 0 [ Hββ Hβc ] 0 0 u β i u β i σ 0 [ Hββ Hβc ] β i= 0 1 u β i > 0. Therefore, for both cases, we always conclude that the second equality in 59 does not hold, which contradicts the assumption. Therefore, we know that H CX; θx if and only if iii holds. ForthegivenS θx, letaffcx; θxbetheaffinehullofthecriticalconecx; θx, i.e., the smallest affine space containing CX; θx. Note that it follows from 57 that 0 CX; θx. It is easy to see cf. e.g., [33, Theorem.7] that affcx; θx = CX; θx CX; θx. Therefore, by Proposition 5.1, one can easily derive the following proposition on the characterization of affcx; θx. For simplicity, we omit the detail proof here. Proposition 5. Suppose that X,S IR m n IR m n is a solution of the GE 3. Let X = X S admit the SVD 7. Then, H affcx; θx if and only if H satisfies the following conditions. 18

19 i If σ k X > 0, then there exists some τ IR such that SU T β HV β = SU T β 1 HV β τi β SU T β 3 HV β3 ii If σ k X = 0 and S = k, then there exists some τ IR such that [ ] SU T U T β HV β U T β β HV 1 HV β = 0 τi β U T b HV b U T b HV iii If σ k X = 0 and S < k, then [ ] U T β HV β U T β HV = SUT β 1 HV β Next, consider the dual GE 4. The critical cone of K at X,t = X S,θX 1 IR m n IR, associated with the complementarity problem in 5, is defined as CX,t;K = T K S, 1 X,θX Thus, we know from 39 that [ τik 0 Ĥ,τ Ĥ = H U CX,t;K 0 0 H CX; θ S, where CX; θ S IR m n is defined by ] V T, 67 CX; θ S := { { H IR m n ϑ S;H X,H = 0 } if ϑs = 1, IR m n if ϑs < We call CX; θ S the critical cone of θ S = N Bk S at X = X S, associated with the dual GE in 4. The following characterization of the critical cone CX; θ S can be obtain similarly as that of CX; θx. For simplicity, we omit the detail proof here. Proposition 5.3 Suppose that X,S IR m n IR m n is a solution of the dual GE 4. Assume that ϑs = 1. Let X = X S admit the SVD 7. Then, H CX; θ S ϑ S;H = X,H = 0, which is equivalent to the following conditions. 19

20 i If σ k X > 0, then tru T βhv β = 0, [ SU T 0 0 α β 1 HV α β1 = 0 SU T β 1 HV β1 ] with SU T β 1 HV β1 0 and [ ] [U T β 3 γhv β3 γ U T β 3 γhv = SU T β 3 HV β ] with SU T β 3 HV β3 0. ii If σ k X = 0 and S < k, then [ SU T 0 0 α β 1 HV α β1 = 0 SU T β 1 HV β1 ] with SU T β 1 HV β1 0. iii If σ k X = 0 and S = k, then tru T β 1 β HV β1 β [ U T b HV b U T b HV ] 0, [ ] SU T 0 0 α β 1 HV α β1 = 0 SU T with SU T β β 1 HV β1 1 HV β1 0. ForthegivenX θ S, letaffcx; θ SbetheaffinehullofthecriticalconeCX; θ S. Therefore, by Proposition 5.3, we obtain the following characterization of affcx; θ S. Proposition 5.4 Suppose that X,S IR m n IR m n is a solution of the dual GE 4. Assume that ϑs = 1. Let X = X S admit the SVD 7. Then, H affcx; θ S if and only if H satisfies the following conditions. i If σ k > 0, then [ SU T 0 0 α β 1 HV α β1 = 0 SU T β 1 HV β1 ], tru T βhv β = and [ [ ] U T β 3 γ HV β 3 γ U T β 3 γ HV = SU T β 3 HV β ]. 70 ii If σ k = 0, then SU T α β 1 HV α β1 = [ SU T β 1 HV β1 ]. 71 0

21 6 The second order analysis In this section, we shall study another important variational property of the Ky Fan k-norm θ = k, i.e., the conjugate function of the parabolic second order directional derivative of θ, which equals to the support function of the second order tangent set of the epigraph of θ. This conjugate function is closely related to the second order optimality conditions of the problem 1. For the given X,θX K, let T i, K X,θX;H,τ and T K X,θX;H,τ be the inner and outer second order tangent sets [, Definition 3.8] to K at X,θX K along the direction H,τ T K X,θX, respectively, i.e., and T i, K X,θX ρh,τ K X,θX;H,τ := liminf ρ 0 1 ρ T K X,θX;H,τ := limsup ρ 0 K X,θX ρh,τ 1, ρ where lim sup and lim inf are the Painlevé-Kuratowski outer and inner limit for sets cf. [34, Definition 4.1]. For TK i, := TK X,θX;H,τ or T K X,θX;H,τ, since K is convex, we know from [, Proposition 3.34, 3.6 & 3.63] that for any X,θX K and H,τ T K X,θX, TK T TK X,θX H,τ T K T TK X,θXH,τ, 7 where T TK X,θX H,τ is the tangent cone of T KX,θX at H,τ. For any given H,τ T K X,θX, let us consider the following two cases. Case 1. k σ i X;H = τ, i.e., H,τ bdt KX,θX. Since intk and the continuous convex function θ = k is parabolically second order directionally differentiable, we know from [, Proposition 3.30] that X,θX;H,τ = T K X,θX;H,τ = epiθ X;H,, T i, K where epiθ X;H, is the epigraph of the parabolic second order directional derivative of θ at X along the direction H, which is convex and given by epiθ X;H, := { W,η IR m n IR k σ i X;H,W η }. 73 Case. k σ i X;H < τ, i.e., H,τ intt KX,θX. Since T TK X,θXH,τ = IR IR m n, we know from 7 that T i, K X,θX;H,τ = T K X,θX;H,τ = IR IR m n. 74 Therefore, we may denote TK X,θX;H,τ the second order tangent set to K at X,θX along the direction H,τ T K X,θX. Next, we shall provide the explicit formula of the support function of the second order tangent set TK X,θX;H,τ. Let X,θX K be fixed. For any H,τ TK X,θX, denote 1

22 T H,τ := TK X,θX;H,τ. Consider the support function δ T τ,h, : IR IRm n, ], i.e., δ T H,τ S,ζ = sup{ S,W ζη W,η T H,τ }, S,ζ IR m n IR. Claim 1 δ T H,τ S,ζ if S,ζ / T TK X,θX H,τ. Proof. LetS,ζ / T TK X,θX H,τ bearbitrarilygiven. SinceTTK X,θX H,τisnonempty, we may assume that there exists W,η T TK X,θXH,τ such that S,ζ,W,η > 0. Fix any η, W T H,τ. By 7, we have for any ρ > 0, Therefore, we know that ρw,η W, η T TK X,θX H,τT H,τ T H,τ. ρ S,ζ,W,η S,ζ, W, η δ S,ζ T H,τ. Since S,ζ,W,η > 0andρ > 0canbearbitrarilylarge, weconcludethatδ T H,τ S,ζ for any S,ζ / T TK X,θX H,τ. Since K is a closed convex cone in IR m n IR, it can be verified easily that K T K X,θX T TK X,θX H,τ. Inparticular,wehave±X,θX T K X,θX T TK X,θX H,τand±H,τ T T K X,θX H,τ. Therefore, we know from the definition of the polar cone that if S,ζ T TK X,θX H,τ, then S,ζ K, S,ζ,X,θX = 0 and S,ζ,H,τ = Hence, by Claim 1, we only need to consider the point S,ζ IR IR m n satisfying the condition 75, since otherwise δt S,ζ. Moreover, instead of considering the general S IR m n, we only consider the point S such that X,S IR m n IR m n satisfying the GE 3, i.e., S θx, which is equivalent to the complementarity problem in 5. On the other hand, by the definition of the critical cone 54 of K, it is evident that the given point S, 1 satisfies the condition 75 if and only if H,τ CX,t;K with X,t = X,θX S, 1. Thus, by 55 and 57, we know that S, 1 satisfies the condition 75 if and only if H CX; θx defined by 56 and τ = S,H = k σ i X;H. Hence, we know from 73 that T H := T H,τ = { W,η IR m n IR k } σ ix;h,w η, 76 where for each i, the second order directional derivative σ i X,H,W is given by Proposition.4. Let X = XS admit the SVD 7. Let a 1,...,a r be the index sets defined by 16 with respect to X. Denote σ = σx and u = σs. Consider the following two cases.

23 Case 1. σ k > 0. Let α, β and γ be the index sets defined by 31 and β 1, β and β 3 be the index sets defined by 35. Let r 1 {0,1,...,r 1}, 1 and r 1 1 r 1 r 1 be the integers such that 36 holds. For each l {1,..., }, since σ i = σ i for any i,i a l, we use ν l to denote the common value. By 55 and 57, we know that there exists an orthogonal matrix R O β such that 60 holds, i.e., Diagu β and SU T βhv β admit asimultaneous ordered eigenvalue decomposition. Therefore, R has the block diagonal structure 61. Hence, we know from the part i of Proposition.4 that W,η T H if and only if k σ ix;h,w = tr trsu T a l WV al tr Ω al X,H R1 T T SU βwv β Ω β X,H R 1 0 i= β 1 1 λ i R T T SU β WV β Ω β X,H R η, 77 where Ω al X,H S m, l = 1,..., and Ω β X,H S m are given by with respect to X, R 1 OSU T β 1 HV β1 and R OSU T β HV β. Meanwhile, since u α = e α, we have for any W,η T H, η S,W = η U T SV,U T WV [ ] [ ] Diaguα 0 SU T = η, αwv α 0 0 Diagu β 0 SU T β WV β where = η = ΞW,η trsu T a l WV T a l Diagu β,su T β WV T β ΞW,η = η Next, we shall show that tr Ω al X,H Diagu β,ω β X,H, 78 trsu T a l WV T a l tr Ω al X,H Diagu β,su T βwv T β Ω β X,H. 79 max { ΞW,η W,η T H } = In fact, since 0 u β e β and e β,u = 0, we know from Lemma.1 Fan s inequality that 3

24 the last term of 79 satisfies tr Diagu β,su T β WV T β Ω βx,h = Diagu β,r T SU T β WV T β Ω βx,h R1 T T SU β WV β Ω β X,H R 1 u β,λ tr R1 T T SU βwv β Ω β X,H R 1 0 i= β 1 1 R T SU T β WV β Ω β X,H R R λ i R T T SU βwv β Ω β X,H R. Therefore, together with 77 and 79, we obtain that for any W,η T H, ΞW,η 0. Also, it is easy to check that there exists W,η T H such ΞW,η = 0. By combining 78 and 80, we obtain that δ T H S, 1 = sup{ S,W η W,η T H } = tr Ω al X,H Diagu β,ω β X,H. 81 Case. σ k = 0. Let α and β be the index sets defined by 34 and β 1, β and β 3 be the index sets defined by 35. Let {0,1,...,r 1} and 1 be the integers such that 37 holds. For each l {1,..., }, since σ i = σ i for any i,i a l, we still use ν l to denote the common value. By 55 and 57, we know that there exist orthogonal matrices E O β and F O β n m such that 63 holds, i.e., [Diagu β 0] and [U T β HV β U T β HV ] admit a simultaneous ordered SVD, which implies that E and F have the block diagonal structure 64. Therefore, we know from the part ii and iii of Proposition.4 that W,η T H if and only if W,η satisfies the following conditions: if σ 0 [U T βhv β U T βhv ] > 0, then = k σ i X;H,W trsu T a l WV T a l tr Ω al X,H tr S Q T 1 [UT β W HX HV β U T β W HX HV ]Q 1 0 i= β 1 1 λ i S Q T [UT β W HX HV β U T β W HX HV ]Q η; 8 4

25 if σ 0 [U T βhv β U T βhv ] = 0, then = k σ i X;H,W trsu T a l WV T a l tr Ω al X,H tr S Q T 1 [UT β W H X HV β U T β W HX HV ]Q 1 0 i= β 1 1 σ i Q T T [U β W HX HV β U T β W H X HV ]Q η, 83 where Ω al X,H S m, l = 1,..., are given by with respect to X, Q 1 O β1, Q O β, Q 3 O b and Q 3 O b n m are given by 64, Q O β β 3 and Q O β b n m are defined by [ ] [ ] Q Q 0 = and Q Q 0 = 0 Q 3 0 Q. 3 Meanwhile, since u α = e α, we have for any W,η T H, η S,W = η U T SV,U T WV [ ] [ ] Diaguα 0 0 SU T α = η, WV α Diagu β 0 0 U T β WV β U T β WV where = η = ΞW,η trsu T a l WV T a l [Diagu β 0],[U T βwv β U T βwv ] tr Ω al X,H [Diagu β 0],[U T βhx HV β U T βhx HV ], 84 ΞW,η = η trsu T a l WV T a l tr Ω al X,H [Diagu β 0],[U T βw HX HV β U T βw HX HV ]. 85 Similarly, we are able to show that max { ΞW,η W,η T H } =

26 In fact, if σ 0 [U T βhv β U T βhv ] > 0, then since 0 u β e β and e β,u β k k 0, we know from Lemma.1 Fan s inequality that the last term of 85 satisfies [Diagu β 0],[U T βw H X HV β U T βw H X HV ] = [Diagu β 0],E T [U T β W HX HV β U T β W HX HV ]F tr S Q T 1 [UT β W H X HV β U T β W HX HV ]Q 1 0 i= β 1 1 λ i S Q T [U T βw HX HV β U T βw HX HV ]Q. Thus, together with 8 and 85, we obtain that ΞW,η 0 for any W,η T H. If σ 0 [U T βhv β U T βhv ] = 0, then by Lemma. von Neumann s trace inequality, we know that [Diagu β 0],[U T βw HX HV β U T βw HX HV ] = [Diagu β 0],E T [U T β W H X HV β U T β W H X HV ]F tr S Q T 1 [UT β W H X HV β U T β W H X HV ]Q 1 0 i= β 1 1 σ i Q T T [U βw HX HV β U T βw HX HV ]Q Together with 83 and 85, we conclude that ΞW,η 0 for any W,η T H. Moreover, it is easy to check that in both case there exists W,η T H such that Ξη,W = 0 e.g., W = HX H IR m n and η = k σ i X;H,W. By combining 84 and 86, we obtain that δ T H S, 1 = sup{ S,W η W,η T } = tr Ω al X,H Diagu β,u T βhx HV β. 87 We summarize the above results on the support function δt H of the second order tangent set T H in the following proposition. Proposition 6.1 Let X,S IR m n IR m n be a solution of the GE 3, i.e., S θx. Let X = X S admit the SVD 7. Denote σ = σx and u = σs. For any H CX, θx, let T H IR m n IR be the second order tangent set defined by 76, and Ω al X,H S m, l = 1,..., and Ω β X,H S m be the matrices given by with respect to X. Then, the support function of T H at S, 1 is given as follows. i If σ k > 0, then δt H S, 1 = tr Ω al X,H Diagu β,ω β X,H.. 6

27 ii If σ k = 0, then δt H S, 1 = tr Ω al X,H Diagu β,u T βhx HV β. Remark 6.1 By 76, we know that for the given S θx and H CX, θx, the second order tangent set T H is the epigraph of the closed convex function ψ := θ X;H, : IR m n IR. Then, the support function of T H at S, 1 obtained in Proposition 6.1 equals to the conjugate function value of ψ at S, i.e., ψ S := sup{ W,S ψw W IR m n } = δ T H S, 1. Definition 6.1 For any given X IR m n, define the function Υ X : θx IR m n IR by for any S θx and H IR m n, if σ k > 0, then if σ k = 0, then Υ X S,H := tr Ω al X,H Diagu β,ω β X,H, Υ X S,H := tr Ω al X,H Diagu β,u T β HX HV β, where σ = σx, u = σs, and Ω al X,H S m, l = 1,..., and Ω β X,H S m are given by with respect to X. Similarly, for the dual GE 4, by employing the similar arguments, we are able to derive the general results on the support function values corresponding to the second order tangent sets of the polar cone K. In particular, we are interesting in the support function value of the following the special second order tangent set T H at H CX; θ S, which is defined by T H := T K S, 1;H,0 = { epiϑ S;H, if ϑs = 1, IR m n IR if ϑs < 1, where ϑ = k is the dual norm of the Ky Fan k-norm. For simplicity, we omit the detail proof here. Proposition 6. Let X,S IR m n IR m n be a solution of the dual GE 4. Suppose that X = X S has the SVD 7. Denote σ = σx and u = σs. For any H CX; θ S, let Ω α β1 S,H S α β 1 and Ω al S,H S al, l = 1,..., r 1 be the matrices defined by with respect to S. Then, the support function of T H at X,θX is given as follows. i If σ k > 0, then r0 δ T H X,θX = ν l tr Ω α β1 S,H σ alal k tr Ω α β1 S,H β1β1 σ k r 1 l= 1 tr Ω al S,H σ k tr U T β 3 HS HV β3 Diagσ γ,u T γ HS HV γ. 7

28 ii If σ k = 0, then δ T H X,θX = r0 ν l tr Ω α β1 S,H. alal Definition 6. For any given S IR m n, define the function Υ S : θ S IR m n IR by for any X θ S and H IR m n, if σ k X > 0, then Υ S X,H := ν l tr Ω α β1 S,H σ alal k tr Ω α β1 S,H β1β1 σ k r 1 l= 1 tr Ω al S,H σ k tr U T β3 HS HV β3 Diagσ γ,u T γ HS HV γ, if σ k X = 0, then Υ S X,H := ν l tr Ω α β1 S,H, alal where σ = σx, u = σs, and Ω α β1 S,H S α β 1 and Ω al S,H S a l, l = 1,..., r 1 are given by with respect to S. It seems that the functions Υ X and Υ are quite complicate from the definitions. However, S one can easily compute the values by elementary calculations. Moreover, we have the following interesting proposition on the defined functions Υ X and Υ. S Proposition 6.3 Let S θx or equivalently X θ S be given. Then, for any H IR m n, Υ X S,H 0, Υ S X,H 0. Moreover, we have Υ X S,H = 0 Υ S X,H = 0, which is equivalent to the following conditions. i If σ k X > 0, then H αα Hαβ1 Hαβ H β1 α H β1 β 1 Hβ1 β S α β 1 β, H β α H β β 1 Hβ β H β1 β 3 = H β3 β 1 T, Hβ β 3 = H β3 β T H αβ = H β α T = 0, Hαβ3 = H β3 α T = 0, H αγ = H γα T = 0, H β1 γ = H γβ1 T = 0, Hβ γ = H γβ T = 0, H αc = 0, Hβ1 c = 0, Hβ c = 0, 88 where H = U T HV, and the index sets α, β, γ, and β i, i = 1,,3 are defined by 31 and 35. 8

29 ii If σ k X = 0, then H αα S α, Hαβ1 = H β1 α T H αβ = H β α T = 0, Hαβ3 = H β3 α T = 0, H αc = 0, 89 where H = U T HV, and the index sets α, β, and β i, i = 1,,3 are defined by 34 and 35. Proof. Let X = XS admit the SVD 7. Denote σ = σx and u = σs. Let H 1 = U T HV 1 and H = U T HV. Consider the following two cases. Case 1. σ k > 0. By and the definition of the pseudoinverse, we obtain that tr Ω al X,H = r1 l =1 l l r1 S H 1 al a ν l ν l l l =1 1 ν l H al, l = 1,..., ν l ν l T H 1 al a l and µ l tr Ω al X,H = l =1 µ l ν l σ k S H 1 al a l r1 l =r 1 1 µ l ν l σ k S H 1 al a l r1 µ l T H 1 al a ν l σ l µ l H al, l = 1,..., r 1. k σ k l =1 Thus, since u i = 0 if i β 3, we have Diaguβ,Ω β X,H = r 1 l= 1 µ l tr Ω al X,H. Therefore, we obtain the following explicit formula of Υ X S,H: Υ X S,H = r 1 l = 1 1 µ l σ k ν l S H 1 al a l r1 l = r 1 1 r1 l =1 ν l ν l S H 1 al a l ν l ν l T H 1 al a l 1 ν l H al r 1 l= 1 9 r 1 r1 l= 1l =r 1 1 r 1 r1 l= 1l =1 µ l ν l σ k S H 1 al a l µ l ν l σ k T H 1 al a l µ l σ k H al. 90

30 Since σ k < ν l, l = 1,...,, ν l < σ k, l = r 1 1,...,r 1, µ l < 1, l = 1,..., r 1, ν l < ν l, l = 1,...,, l = r 1 1,...,r 1, ν l > 0, l = 1,..., r 1, it is easy to see that all the coefficients of the quadric terms of 90 are negative, which implies that Υ X S,H 0 and Υ X S,H = 0 if and only if H IR m n satisfies the conditions 88. Meanwhile, by and the pseudoinverse, we obtain that and ν l tr Ω α β1 S,H alal = σ k tr Ω al S,H l =1 = σ k 1 µ l S H 1 al a l r 1 l = 1 ν l µ l 1 S H 1 al a l r1 l = r 1 1 ν l 1 S H 1 al a l r1 ν l µ l 1 T H 1 al a l ν l 1 H al, l = 1,..., l =1 r 1 l = 1 l l σ k µ l µ l S H 1 al a l r1 l = r 1 1 r1 σ k T H 1 al a µ l µ l σ k H al, l = 1,..., r 1. l µ l l =1 σ k µ l S H 1 al a l NotethatforanyA,B IR p q, tra T B = AB A B. Thus, wehaveforl = r 1 1,...,r 1, σ k tr U T al HS HV al = l =1 σ k 1 and for l = r 1 1,...,r, ν l tr U T al HS HV al = l =1 ν l 1 S H 1 al a l T H 1 al a l S H 1 al a l T H 1 al a l r 1 l = 1 r 1 l = 1 By noting that σ i = 0 if i b, we have σ k tr U T β 3 HS HV β3 Diagσ γ,u T γhs HV γ = r 1 l= r 1 1 σ k tr U T al HS HV al 30 r l=r 1 1 σ k µ l ν l µ l 91 S H 1 al a l T H 1 al a l, S H 1 al a l T H 1 al a l. ν l tr U T al HS HV al.