Variational Analysis of the Ky Fan k-norm

Transcription

1 Set-Valued Var. Anal manuscript No. will be inserted by the editor Variational Analysis of the Ky Fan k-norm Chao Ding Received: 0 October 015 / Accepted: 0 July 016 Abstract In this paper, we will study some variational properties of the Ky Fan k-norm θ = k of matrices, which are closed related to a class of basic nonlinear optimization problems involving the Ky Fan k-norm. In particular, for the basic nonlinear optimization problems, we will introduce the concept of nondegeneracy, strict complementarity and the critical cones associated with the generalized equations. Finally, we present the explicit formulas of the conjugate function of the parabolic second order directional derivative of θ, which will be referred to as the sigma term of the second order optimality conditions. The results obtained in this paper provide the necessary theoretical foundations for future work on sensitivity and stability analysis of the nonlinear optimization problems involving the Ky Fan k-norm. Keywords Ky Fan k-norm nondegeneracy critical cone second order tangent sets Mathematics Subject Classification K10 90C5 90C33 1 Introduction Let R m n be the vector space of all m n real matrices equipped with the inner product Y, Z := TrY T Z for Y and Z in R m n, where Tr denotes the trace, i.e., the sum of the diagonal entries, of a squared matrix. For simplicity, we always assume that m n. For any given positive integer 1 k m, denote θ := k the matrix Ky Fan k-norm, i.e., the sum of k largest singular values of matrices. In particular, 1 coincides with the spectral norm of the matrices, i.e., the largest singular value of matrices; m is the nuclear norm of matrices, i.e., the sum of singular values of matrices. It is well-known that ϑz := Z k = max Z, Z /k for Z R m n is the dual norm of k cf. [1, Exercise IV.1.18]. Since θ is a matrix norm convex, closed, positively homogeneous and θ0 = 0, we obtain from [33, Theorem 13.5 & 13.] that the conjugate function θ = δ θ0 is just the indicator function of the subdifferential θ0 of θ at 0. This work is supported in part by the National Natural Science Foundation of China Grant No This work was initiated while C. Ding was with Department of Mathematics, National University of Singapore during 007 to 01. C. Ding Institute of Applied Mathematics, Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing, P.R. China. Tel.: dingchao@amss.ac.cn

2 Chao Ding Moreover, it can be verified directly from the definition of dual norm that θ0 coincides with the unit ball under the dual norm ϑ, i.e., θ0 = B k := S R m n ϑs 1. Consider the following nonlinear optimization problem involving the Ky Fan k-norm θ = k min fx θgx x X, 1 where X is a finite dimensional real vector space equipped with a scalar product,, f : X R is a continuously differentiable real value function, and g : X R m n is a continuously differentiable function. Since θ is convex and finite everywhere, it is well-known [34, Example 10.8] that for a locally optimal solution x X of 1, there always exists a Lagrange multiplier S R m n, together with x satisfying the following first order optimality condition, namely the Karush-Kuhn-Tucker KKT condition: f x g x S = 0 and S θx, where X := g x, f x X is the gradient of f at x, g x : R m n X is the adjoint of the derivative mapping g x. Note that if X = R m n and gx := x is the identity mapping, then the KKT condition becomes the following generalized equation: 0 fx θx. Note also that if the function θ is replaced by the indicator function δ K of a set K in a finite dimensional real vector space, then the nonlinear optimization problem 1 becomes min fx s.t. gx K. 3 During the last three decades, considerable progress has been made in the variational analysis related to the problem 3 [34,1,,19,4]. In particular, for the general non-polyhedral set K e.g., the second-order cone and the positively semidefinite SDP matrices cone, by employing the well studied properties of the variational inequality S N K x, some important properties of 3, such as the constraint nondegeneracy, second order optimality conditions, strong regularity, full stability and calmness, are studied recently by various researchers [, 35, 5]. In order to extend those results to the optimization problems involving the Ky Fan k-norm, we need first study the variational properties of 1, especially the properties of the generalized equation S θx and its equivalent dual problem X θ S. Although the optimization problem 1 seems extremely simple, many fundamental and important issues such as the concept of nondegeneracy, the characterizations of critical cones and the second order optimality conditions, are not studied yet in literature. The main purpose of this paper is to build up the necessary variational foundations for the future work on the nonlinear optimization problems involving the Ky Fan k-norm. Certainly, instead of the basic model 1, one can consider its various modifications, e.g., the nonlinear optimization problems involving the Ky Fan k-norm with equality and conic constraints. In particular, the following convex composite matrix optimization problems involving the Ky Fan k-norm frequently arise in various applications such as the matrix norm approximation, matrix completion, rank minimization, graph theory, machine learning, etc [15,36,37,9,4,5,6,7,40,8,3, 14,,11,17,3]: min 1 X, Y, QX, Y C, X, Y θx 4 s.t. AX, Y = b, Y K, where Y is a finite dimensional real vector space, Q : R m n Y R m n Y is a positively semidefinite self-adjoint linear operator, A : R m n Y R p is a linear operator, C R m n Y and b R p are given data, and K Y is a closed convex cone e.g., the positive orthant, secondorder cone of vectors, positive semidefinite matrices cone. As the initial step, in this paper, we will mainly focus on the fundamental model 1, since the obtained variational results will provide the necessary theoretical foundations for the study of more complicate model, e.g., 4. More

3 Variational Analysis of the Ky Fan k-norm 3 precisely, we will study the concepts of nondegeneracy and strict complementary to locally optimal solutions of 1. Also, we will define and provide the complete characterizations of the critical cones associated with the generalized equation S θx and its dual problem X θ S. Another important variational property studied in this paper is the conjugate function of the parabolic second order directional derivative of the Ky Fan k-norm θ, which equals to the support function of the second order tangent set of the epigraph of θ. This conjugate function is closely related to the second order optimality conditions of the problem 1. Note that the epigraph of θ is not polyhedral. In general, the conjugate function of the parabolic second order directional derivative of the Ky Fan k-norm θ will not vanish in the corresponding second order optimality conditions, and will be referred to as the sigma term, provides the second order information of θ. In this paper, we provide the explicit expression of this sigma term. Consequently, it becomes possible to establish the second order optimality conditions of the problem 1 and study many corresponding sensitivity properties, e.g., the second order optimality conditions and the characterization of strong regularity of the KKT solutions. The remaining parts of this paper are organized as follows. In Section, we introduce some preliminary results on the differential properties of eigenvalue values and vectors of symmetric matrices and singular values and vectors of matrices. In Section 3, we study the properties of the solution of the GE S θx, which arises from the KKT condition and its equivalent dual form X θ S. We introduce the nondegeneracy and strict complementarity of 1 in Section 4. In Section 5, we introduce and study the critical cones associated with the GE S θx and X θ S. The second order properties of the Ky Fan k-norm θ are studied in Section 6. We conclude our paper in the final section. Below are some common notations to be used: For any Z R m n, we denote by Z ij the i, j-th entry of Z. For any Z R m n, we use z j to represent the jth column of Z, j = 1,..., n. Let J 1,..., n be an index set. We use Z J to denote the sub-matrix of Z obtained by removing all the columns of Z not in J. So for each j, we have Z j = z j. Let I 1,..., m and J 1,..., n be two index sets. For any Z R m n, we use Z IJ to denote the I J sub-matrix of Z obtained by removing all the rows of Z not in I and all the columns of Z not in J. We use to denote the Hardamard product between matrices, i.e., for any two matrices X and Y in R m n the i, j-th entry of Z := X Y R m n is Z ij = X ij Y ij. Preliminaries In this section, we list some useful preliminary results on the eigenvalues of symmetric matrices and the singular values of matrices, which are useful for our subsequent analysis. Let S n be the space of all real n n symmetric matrices and O n be the set of all n n orthogonal matrices. Let X S n be given. We use λ 1 X λ X... λ n X to denote the real eigenvalues of X counting multiplicity being arranged in non-increasing order. Denote λx := λ 1 X, λ X,..., λ n X T R n and ΛX := diagλx, where for any x R n, diagx denotes the diagonal matrix whose i-th diagonal entry is x i, i = 1,..., n. Let P O n be such that X = P ΛXP T. 5 We denote the set of such matrices P in the eigenvalue decomposition 5 by O n X. Let ω 1 X > ω X >... > ω r X be the distinct eigenvalues of X. Define the index sets a k := i λ i X = ω k X, 1 i n, k = 1,..., r. 6

4 4 Chao Ding For each i 1,..., n, we define l i X to be the number of eigenvalues that are equal to λ i X but are ranked before i including i and s i X to be the number of eigenvalues that are equal to λ i X but are ranked after i excluding i, respectively, i.e., we define l i X and s i X such that λ 1 X... λ i li XX > λ i li X1X =... = λ i X =... = λ isi XX > λ isi X1X... λ n X. 7 In later discussions, when the dependence of l i and s i, i = 1,..., n, on X can be seen clearly from the context, we often drop X from these notations. Next, we list some useful results about the symmetric matrices which are needed in subsequent discussions. The inequality in the following lemma is known as Fan s inequality [13]. Lemma 1 Let Y and Z be two matrices in S n. Then Y, Z λy T λz. 8 where the equality holds if and only if Y and Z admit a simultaneous ordered eigenvalue decomposition, i.e., there exists an orthogonal matrix U O n such that Y = UΛY U T and Z = UΛZU T. The following proposition on the directional differentiability of the eigenvalue function λ is well known. For example, see [0, Theorem 7] and [38, Proposition 1.4]. Proposition 1 Let X S n have the eigenvalue decomposition 5. Then, for any S n H 0, we have λ i X H λ i X λ li P T a k HP ak = O H, i α k, k = 1,..., r, 9 where for each i 1,..., n, l i is defined in 7. Hence, for any given direction H S n, the eigenvalue function λ i is directionally differentiable at X with λ ix; H = λ li P T a k HP ak, i a k, k = 1,..., r. Let k 1,..., r be fixed. For the symmetric matrix Pa T k HP ak S ak, consider the eigenvalue decomposition Pa T k HP ak = RΛPa T k HP ak R T, 10 where R O ak. Denote the distinct eigenvalues of Pa T k HP ak by µ 1 > µ >... > µ r. Define ã j := i λ i P T a k HP ak = µ j, 1 i a k, j = 1,..., r. 11 For each i a k, let l i 1,..., a k and k 1,..., r be such that li := l li P T a k HP ak and l i ã k, 1 where l i is defined by 7. Let X and X be two finite dimensional real Euclidean spaces. We say that a function Φ : X X is parabolic second order directionally differentiable at x X, if Φ is directionally differentiable at x and for any h, w X, Φx th 1 lim t w Φx tφx; h t 0 1 exists; t and the above limit is said to be the parabolic second order directional derivative of Φ at x along the directions h and w, denoted by Φ x; h, w. The following proposition [38, Proposition.], provides the explicit formula of the parabolic second order directional derivative of the eigenvalue function.

5 Variational Analysis of the Ky Fan k-norm 5 Proposition Let X S n have the eigenvalue decomposition 5. Then, for any given H, W S n, we have for each k 1,..., r ] λ i X; H, W = R T λ li ã kp a T k [W HX λ i I n H P ak, i a Rã k k, 13 where Z R n n is the Moore-Penrose pseudoinverse of the square matrix Z R n n. Let X R m n be given. Without loss of generality, assume that m n. We use σ 1 X σ X... σ m X to denote the singular values of X counting multiplicity being arranged in non-increasing order. Define σx := σ 1 X, σ X,..., σ m X T and ΣX := diagσx. Let X R m n admit the following singular value decomposition SVD: X = U [ΣX 0] V T = U [ΣX 0] [V 1 V ] T = UΣXV T 1, 14 where U O m and V = [V 1 V ] O n with V 1 R n m and V R n n m. The set of such matrices pair U, V in the SVD 14 is denoted by O m,n X, i.e., O m,n X := U, V O m O n X = U [ΣX 0] V T. Define the three index sets a, b and c by a := i σ i X > 0, 1 i m, b := i σ i X = 0, 1 i m and c := m 1,..., n. 15 Let ν 1 X > ν X >... > ν r X > 0 be the distinct nonzero singular values of X. Without causing any ambiguity, we also use a k to denote the following index sets a k := i σ i X = ν k X, 1 i m, k = 1,..., r. 16 For the sake of convenience, let a := b. For each i 1,..., m, we also define l i X to be the number of singular values that are equal to σ i X but are ranked before i including i and s i X to be the number of singular values that are equal to σ i X but are ranked after i excluding i, respectively, i.e., we define l i X and s i X such that σ 1 X... σ i li XX > σ i li X1X =... = σ i X =... = σ isi XX > σ isi X1X... σ m X. 17 In later discussions, when the dependence of l i and s i, i = 1,..., m, on X can be seen clearly from the context, we often drop X from these notations. The inequality in the following lemma is known as von Neumann s trace inequality [7]. Lemma Let Y and Z be two matrices in R m n. Then Y, Z σy T σz, 18 where the equality holds if Y and Z admit a simultaneous ordered singular value decomposition, i.e., there exist orthogonal matrices U O m and V O n such that Y = U[ΣY 0]V T and Z = U[ΣZ 0]V T. by For notational convenience, define two linear operators S : R p p S p and T : R p p R p p SZ := 1 Z ZT and T Z := 1 Z ZT Z R p p. 19 The following proposition on the directional derivatives of the singular value functions can be obtained directly from Proposition 1. For more details, see [1, Section 5.1].

6 6 Chao Ding Proposition 3 Let X R m n have the singular value decomposition 14. For any R m n H 0, we have σ i X H σ i X σ ix; H = O H, i = 1,..., m, 0 with σ ix; H = λli SU T ak HV ak if i a k, k = 1,..., r, σ li [U T b HV b U T b HV ] if i b, 1 where for each i 1,..., m, l i is defined in 17. Similarly, one can derive the following explicit formulas of the parabolic second order directional derivatives of the singular value functions from Proposition, directly. For more details, see [4, Theorem 3.1]. Proposition 4 Let X R m n have the singular value decomposition 14. Suppose that the direction H, W R m n are given. i If σ i X > 0, then σ i X; H, W = R λ li T α k SUa T k W V ak Ω ak X, H R α k, where k 1,..., r such that i a k, Ω ak X, H S m is given by Ω ak X, H = SU T HV 1 ak T ΣX ν k XI m SU T HV 1 ak T U T HV 1 ak T ΣX ν k XI m T U T HV 1 ak 1 ν k X U T a k HV V T H T U ak, the matrix R O ak satisfies SUa T k HV ak = RΛSUa T k HV ak R T, and α r j j=1 and l i, k be defined by 11 and 1 respectively for SUa T k HV ak. ii If σ i X = 0 and σ li [Ub T HV b Ub T HV ] > 0, then σ i X; H, W = λ li SE T ã k[u T b ZV b U T b ZV ]Fã k, where Z = W HX H R m n, X R n m is the Moore-Penrose pseudoinverse of X R m n, E O b, F = [F 1 F ] O b n m satisfy [U T b HV b U T b HV ] = E[Σ[U T b HV b U T b HV ] 0]F T, li 1,..., ã k and k 1,..., r such that l i = l li SE ã k[u T b T ZV b ã j, j = 1,..., r are the index sets of [Ub T HV b Ub T HV ] defined by U T b ZV ]Fã k and l i ã k, ã j := i σ i [U T b HV b U T b HV ] = ν j, 1 i b, and ν 1 > ν >... > ν r are the nonzero distinct singular values of [Ub T HV b Ub T HV ]. iii If σ i X = 0 and σ li [Ub T HV b Ub T HV ] = 0, then σ i X; H, W = E T b [U T σ li b ZV b Ub T ZV ][F b F ], where Z = W HX H R m n, b := i σ i [Ub T HV b Ub T HV ] = 0, 1 i b and l i = l li E T b [Ub T ZV b Ub T ZV ][F b F ] is defined by 17 with respect to E T b [Ub T ZV b Ub T ZV ][F b F ].

7 Variational Analysis of the Ky Fan k-norm 7 3 The generalized equations In this section, we first study some properties of the following simple generalized equation GE 0 S θx, 3 which is equivalent to the following dual form 0 X θ S. 4 Since θ = δ Bk, it follows from [33, Theorem 3.5] that 3 and 4 are also equivalent to the following complementarity problem X, θx K, S, 1 K and X, θx, S, 1 = 0, 5 where K is the epigraph of θ = k, i.e., K = epi θ = X, t R m n R t X k 6 and K is the polar cone of K given by K = ρ θ0, 1 = epi ϑ with ϑ = k. ρ 0 On the other hand, it is well-known [6] see also [33, Theorem 31.5] that X, S is a solution of the GE 3 or 4 if and only if X Pr θ X S = 0 S Pr θ X S = 0, where Pr θ : R m n R m n is the Moreau-Yosida proximal mapping of θ, and Pr θ : R m n R m n is the Moreau-Yosida proximal mapping of θ. Denote X := XS. Let X admit the following singular value decomposition X = U [ΣX 0] V T. 7 Let σ = σx, σ = σx and u = σs be the singular values of X, X and S, respectively. Since and k are unitarily invariant, we know from von Neumann s trace inequality Lemma that X = U [Diag σ 0] V T and S = U [Diag u 0] V T with σ = gσ and u = σ gσ, 8 where g : R m R m is the Moreau-Yosida proximal mapping of the vector k-norm i.e., the sum of the k largest components in absolute value of any vector in R m. The properties of the proximal mapping g have been studied recently in [41], e.g., for any given x R m, the unique optimal solution gx R m can be computed within Om arithmetic operations see [41, Section 3.1] for details. The following simple observations are useful for our subsequence analysis, which can be obtained directly from the characterization of the subdifferential of θ = k cf. [39,8]. Lemma 3 σ and u are the singular values of the solution X, S of the GE 3 or 4 if and only if σ and u satisfy the following conditions. i If σ k > 0, then u α = e α, 0 u β e β, where 0 k 0 k 1 and k k 1 m are two integers such that and u i = k k 0 and u γ = 0, 9 i β σ 1... σ k0 > σ k0 1 =... = σ k =... = σ k1 > σ k σ m 0 30 α = 1,..., k 0, β = k 0 1,..., k 1 and γ = k 1 1,..., m. 31

8 8 Chao Ding ii If σ k = 0, then u α = e α, 0 u β e β and where 0 k 0 k 1 is the integer such that u i k k 0, 3 i β σ 1 σ k0 > σ k0 1 =... = σ k =... = σ m = 0 33 and α = 1,..., k 0 and β = k 0 1,..., m. 34 For notational convenience, we use β 1, β and β 3 to denote the index sets β 1 := i β u i = 1, β := i β 0 < u i < 1 and β 3 := i β u i = For X = XS, let a, b and c be the index sets defined by 15. We use a 1,..., a r to denote the index sets defined by 16 with respect to X and a = b for the sake of convenience. Thus, by Lemma 3, we know that if σ k > 0, then there exist integers r 0 r 1 0, 1,..., r 1, r 0 r 0 r 0 1 and r 1 1 r 1 r 1 such that α = r 0 a l, β 1 = r 0 l=r 0 1 a l, β = r 1 l= r 0 1 a l, β 3 = r 1 l= r 1 1 a l and γ = l=r 1 1 if σ k = 0, then there exist integers r 0 0, 1,..., r 1 and r 0 r 0 r 0 1 such that α = r 0 a l, β 1 = r 0 l=r 0 1 a l, β = r l= r 0 1 a l ; 36 a l and β 3 = b. 37 Moreover, we know that for each l 1,..., r 0, σ i = σ j for any i, j a l, which implies that we can use ν 1 >... > ν r0 > 0 to denote those common values. Similarly, if σ k > 0, we use µ r0 1 >... > µ r 1 > 0 to denote the corresponding common values of u; if σ k = 0, we use µ r0 1 >... > µ r > 0 to denote the corresponding common values of u. 4 The nondegeneracy and strict complementarity In this section, we shall introduce the nondegeneracy and strict complementarity of the optimization problem 1. To do so, let us consider the following conic reformulation of 1: min fx t s.t. gx, t K, 38 where K = epi θ. Let x, t be a feasible point of 38. Denote X = g x R m n. Recall the definition [34, Definition 6.1] of the tangent cone T K X, t of K at the given point X, t K, i.e., T K X, t = H, τ R m n R ρ n 0, dist X, t ρ n H, τ, K = oρ n. For any convex function φ : R m n,, we know from [9, Theorem.4.9] that T epi φ Y, φy = epi φ Y ; := H, τ R m n R φ Y ; H τ, Y R m n. 39

9 Variational Analysis of the Ky Fan k-norm 9 Therefore, for θ = k, we know from Proposition 3 that 0 T H, τ tru αhv α λ i SU T β HV β τ if σ k X > 0, T K X, θx = 0 [ ] T H, τ tru αhv α σ i U T β HV β U T β HV τ if σ k X = Define G : X R R m n R by Gx, t := gx, t, x, t X R. Robinson s CQ [30] for 38 at a given feasible point x, t can be written as G x, tx R T K X, t = R m n R. 41 Proposition 5 For any x X, Robinson s CQ 41 for 38 holds at x, θg x. Proof. Note that the directional derivative θ X; of the Ky Fan k-norm is finite everywhere. Therefore, the results can be derived directly from 41 and 39. In fact, we only need to show that for any given X, t R m n R, there exists h, η X R and H, τ T K X, t with t = θg x such that g xh, η H, τ = X, t. Let H = X and τ = θ X; X. By choosing h = 0 and η = t τ, we know that the above equality holds trivially. As we mentioned in Section 1, for a locally optimal solution x to the optimization problem 1, the corresponding Lagrange multiplier always exists. By 5, we know that there exists a Lagrange multiplier S R m n if and only if there exists S R m n such that x, θg x, S, 1 is the following KKT solution of 38. On the other hand, it is well-known [43] that for a locally optimal solution of 38, the corresponding set of Lagrange multipliers is nonempty, convex, bounded and compact if and only if Robinson s CQ 41 holds. Therefore, the following proposition follows from Proposition 5, directly. Proposition 6 Let x X be a locally optimal solution to the problem 1. The set of Lagrange multipliers of 1 is a nonempty, convex, bounded and compact subset of R m n. Next, let us study the concept of nondegeneracy for the optimization problem 1. For any convex function φ : R m n, and Y R m n, the lineality space of T epi φ Y, φy, i.e., the largest linear subspace in T epi φ Y, φy, can be written as lin T epi φ Y, φy = T epi φ Y, φy T epi φ Y, φy = H, τ R m n R φ Y ; H τ φ Y ; H = H, τ R m n R φ Y ; H = φ Y ; H = τ. 4 The last equation of 4 follows from [33, Theorem 3.1], directly. For the Ky Fan k-norm θ = k, define the linear subspace T lin X R m n by T lin X := H R m n θ X; H = θ X; H. 43 If S θx, then, by Proposition 3, we have H R m n SU T T lin β HV β = τi β for some τ R if σ k X > 0, X = H R m n [ U T β HV β U T ] β HV = 0 if σ k X = 0, 44

10 10 Chao Ding where U O m and V O n are eigenvectors of X = X S, and the index set β is defined in 31 if σ k X > 0 and in 34 if σ k X = 0. For the problem 38, the concept of Robinson s constraint nondegeneracy [31, 3] can be specified as follows. The constraint nondegeneracy for 38 holds at the feasible point x, t if G x, tx R lin T K X, t = R m n R, 45 where the lineality space lin T K X, t is given by 4 with respect to θ = k. Proposition 7 The constraint nondegeneracy 45 for 38 holds at x, θx if and only if g xx T lin X = R m n, 46 where T lin X R m n is the linear subspace defined by 43. Therefore, we say that the nondegeratacy for the problem 1 holds at x if 46 holds. Proof. For any given X R m n, by 45, we know that there exists h X, H, η lin T K X, t such that g xh, η H, η = X, 0. Since H T lin X, we know that 46 holds. Conversely, for any X, t R m n R, by 46, we know that there exists h X and H T lin X such that g xh H = X. Denote τ = θ X; H. By taking η = t τ, we obtain that g xh, η H, τ = X, t, which implies that the constraint nondegeneracy 45 holds at x, θx. Let x X be a locally optimal solution of 1. Denote X = g x. Let β be the index set defined in 31 if σ k X > 0 and in 34 if σ k X = 0. The following definition of the strict complementarity of 1 can be regarded as a generalization of the strict complementarity for the constraint optimization problem cf. [, Definition 4.74]. Definition 1 We say the strict complementarity condition holds at x X if there exists S ri θx such that f x g x S = By Lemma 3, one can derive the following proposition easily. For simplicity, we omit the detail proof here. Proposition 8 The strict complementarity condition holds at x X if and only if there exists S θx such that 47 holds and i if σ k X > 0, then 0 < σ β S < e β ; ii if σ k X = 0, then σ β S < e β and i β σ is < k k 0. In the following proposition, we will show that the nondegeneracy condition 46 implies the uniqueness of the Lagrange multiplier. Proposition 9 Let x X be a locally optimal solution of 1. Denote X = g x. If x is nondegenerate, then S satisfying is unique. Conversely, if S satisfying is unique and the strict complementarity condition holds at x, then x is nondegenerate.

11 Variational Analysis of the Ky Fan k-norm 11 Proof. Suppose that x is nondegenerate and let S and S satisfy. Then, we know that g x S S = 0, which implies that := S S [ g xx ]. Denote X = X S and X = X S. Suppose that X and X admit the SVD: X = U[ΣX 0]V T and X = U [ΣX 0]V T, where U, U O m and V, V O n. By 8, we know that both U, V and U, V are eigenvalue vectors of X. Therefore, it follows from [10, Proposition 5] that if σ k X > 0, then there exist orthogonal matrices Q 1 O α, Q O β, Q 3 O γ and Q 3 O γ n m such that Q U = U 0 Q 0 and V = V 0 0 Q 3 Q Q 0 ; 0 0 Q 3 if σ k X = 0, then there exist orthogonal matrices Q 1 O α, Q O β and Q O β n m such that [ ] [ ] U Q1 0 = U and V Q1 0 = V 0 Q 0 Q. Therefore, by 8, we know from Lemma 3 that if σ k X > 0, then if σ k X = 0, then U T V = 0 SU T β V β 0 0 with trsu T β V β = U T V = Thus, we know from 44 that in both cases, [ U T β V β U T β V ]. 49, H = U T V, U T HV = 0 H T lin X, which implies that [ T lin X ]. Therefore, by 46, we know that = 0, i.e., S satisfying is unique. Conversely, since the strict complementarity condition holds at x, we know that the unique Lagrange multiplier S θx satisfying i or ii of Proposition 8. Let X = X S admit the SVD 7. Suppose that the constraint nondegenerate condition 46 does not hold at X, i.e., there exists 0 H [ g xx ] [ T lin X ]. Therefore, we know that g X H = 0. Moreover, by 44, we know that if σ k X > 0, then 48 holds for H; if σ k X = 0, then 49 holds for H. Since g X H = 0, we know that for any ρ, f x g X S ρh = 0. Moreover, since S satisfies i and ii of Proposition 8, by 48 and 49, we know from Lemma 3 that for ρ > 0 small enough, S ρh θx. This contradicts the uniqueness of S. Remark 1 Let X θ S. For the dual norm ϑ = k, since S, 1 K, we have T K S, 1 = H, τ R m n R ϑ S; H τ if ϑs = 1, R m n R if ϑs < 1.

12 1 Chao Ding We define the linear subspace T lin S R m n by T lin S := H R m n ϑ S; H = ϑ S; H = 0 if ϑs = 1, R m n if ϑs < For the case that ϑs = max S, S /k = 1, we know from Proposition 3 that if S < k, then T lin S = H R m n S[U α U β1 ] T H[V α V β1 ] = 0 ; 51 if S = k, then T lin S = H R m n SU T α β 1 HV α β1 = 0, tru T β HV β = 0, U T β 3 γhv β3 γ c = 0, 5 where U O m and V O n are eigenvectors of X = X S, the index set β is defined in 31 if σ k X > 0 and in 34 if σ k X = 0, and β 1, β and β 3 are the index sets defined by The critical cones From now on, let us always assume that X = g x and S are solutions of the GEs 3 and 4. Therefore, the critical cones associated with the GEs 3 and 4 can be defined correspondingly from the critical cones associated the complementarity problem 5. Firstly, consider the GE 3. Denote X, t = X S, θx 1. The critical cone of K at X, t associated with the complementarity problem in 5, is defined as Thus, we know from 39 that C X, t; K = T K X, θx S, H, τ C X, t; K H CX; θx, τ = S, H, 54 where CX; θx R m n is defined by CX; θx := H R m n θ X; H S, H. 55 Since θ X; is a positively homogeneous convex function with θ X; 0 = 0, CX; θx is indeed a closed convex cone. We call CX; θx the critical cone of θx at X = X S, associated with the GE 3. Next, we present the following proposition on the characterization of the critical cone CX; θx. Proposition 10 Suppose that X, S R m n R m n is a solution of the GE 3. Let X = X S admit the SVD 7. Then, which is equivalent to the following conditions: H CX; θx θ X; H = S, H, 56 i if σ k X > 0, then there exists some τ R such that λ β1 SU T β 1 HV β1 τ λ 1 SU T β 3 HV β3 and SU T β HV β = SU T β 1 HV β τi β 0 ; 0 0 SU T β 3 HV β3

13 Variational Analysis of the Ky Fan k-norm 13 ii if σ k X = 0 and S = k, then there exists some τ 0 such that [U λ β1 SU T T β 1 HV β1 τ σ 1 b HV b U T ] b HV and ] SU [U T T β HV β U T β 1 HV β β HV = 0 τi β 0 0 ; 0 0 U T b HV b U T b HV iii if σ k X = 0 and S < k, then SU T β 1 HV β1 0 and ] [U T β HV β U T β HV = SU T β 1 HV β Proof. Denote σ = σx and u = σs. By S, H = U T SV, U T HV = [Diagu 0], U T HV, we know from Lemma 3 that for any H R m n, tru T αhv α Diagu β, SU T β HV β if σ k > 0, S, H = ] tru T αhv α [Diagu β 0], [U T β HV β U T β HV if σ k = 0. Thus, by combining with Fan s inequality Lemma 1 and von Neumann s trace inequality Lemma, we obtain that for any H R m n, if σ k > 0, and if σ k = 0, Diagu β, S H ββ u T β λs H 0 ββ λ i S H ββ, 57 [Diaguβ 0 ], [ ] [ Hββ Hβc u T ] 0 [ ] β σ Hββ Hβc σ i Hββ Hβc, 58 where H = U T HV. Therefore, we know from 40 that H CX; θx θ X; H = S, H the equalities in 57 and 58 hold. Consider the following two cases. Case 1 σ k > 0. It follows from Lemma 1 that the first equality of 57 holds if and only if Diagu β and S H ββ admit a simultaneous ordered eigenvalue decomposition, i.e., there exists R O β such that Diagu β = RDiagu β R T and S H ββ = RΛS H ββ R T. 59 Let r 0 r 1 0, 1,..., r 1, r 0 r 0 r 0 1 and r 1 1 r 1 r 1 be the integers such that 36 holds. Therefore, the orthogonal matrix R O β has the following block diagonal structure: R R R = 0 R 0 with R = , R R r 1 r 0

14 14 Chao Ding where R 1 O β1, R O β, R 3 O β3 and R l O a r l 0, l = 1,..., r 1 r 0. Thus, 59 holds if and only if S H ββ S β has the following block diagonal structure: S H β1 β S H a r0 1a r S H ββ = S H a r1 a r S H β3 β 3 and the elements of λs H β1 β 1, λs H a r0 1a r0 1,..., λs H a r1 a r1, λs H β3 β 3 are in nonincreasing order and are the eigenvalues of the symmetric matrix S H ββ. On the other hand, by 9, we know that u β1 = e β1, 0 < u β < e β, u β3 = 0 and e β, u β = k k 0. Then, we can verify that the second equality of 57 holds if and only if λ i S H ββ = λ j S H ββ i, j β 1 1,..., β 1 β. 61 In fact, it is clear that 61 implies the second equality of 57 holds. Conversely, without loss of generality, assume that β, then k k 0 β 1 1,..., β 1 β. Suppose that there exists i β 1 1,..., β 1 β but i k k 0 such that λ i S H ββ > λ 0 S H ββ or λ 0 S H ββ > λ i S H ββ. Then, since 0 < u β < e β and e β, u β = k k 0, for both cases, we always have = > λ i S H ββ u T β λs H ββ λ i S H ββ 1 u β i λ 0 S H ββ 1 u β i β i= 0 1 β i= 0 1 = λ 0 S H 0 ββ k k 0 u β i u β i λ i S H ββ, u β i λ 0 S H ββ β i= 0 1 u β i = 0, which implies that the second equality of 57 does not hold, which contradicts the assumption. Therefore, we know that H CX; θx if and only if i holds. Case σ k = 0. We know from Lemma that the first equality of 58 holds if and only if [Diagu β 0] and [ ] Hββ Hβc admit a simultaneous ordered SVD, i.e., there exist orthogonal matrices E O β and F O β n m such that [Diagu β 0] = E[Diagu β 0]F T and [ Hββ Hβc ] = E[Σ[ Hββ Hβc ] 0]F T. 6 Let r 0 0, 1,..., r 1 and r 0 r 0 r 0 1 be the integers such that 37 holds. Therefore, it follows from [10, Proposition 5] that there exist orthogonal matrices Q 1 O β1, Q O β, Q 3 Q β3 and Q 3 O β3 n m such that E = Q Q Q 3 and F = Q Q Q 3 with Q = Q Q r r 0, 63

15 Variational Analysis of the Ky Fan k-norm 15 where Q l O a r 0 l, l = 1,..., r r 0. Thus, 6 holds if and only if [ Hββ Hβc ] has the following block diagonal structure: H ar0 1a r Ha r0 1a r [ ] Hββ Hβc = Har a r Hbb Hbc with H al a l S al, l = r 0 1,..., r, and the elements of h := λ H ar0 1a r0 1, λ H a r0 1a r0 1,..., λ H ar a r, σ[ H bb Hbc ] R m are nonnegative and in non-increasing order and h = σ [ H ββ Hβc ]. On the other hand, by 3, we know that u β1 = e β1, 0 < u β < e β, u β3 = 0 and e β, u β k k 0. Then, we may conclude that the second equality of 58 holds if and only if σi [ Hββ Hβc ] = σ j [ Hββ Hβc ] i, j β 1 1,..., β 1 β if e β, u β = k k 0, σ i [ Hββ Hβc ] = 0 i β 1 1,..., β if e β, u β < k k In fact, it is evident that 64 implies that the second equality of 58 holds. Conversely, consider the following two sub-cases. Case.1 S = k, i.e., e β, u β = 0. Without loss of generality, assume that β, which implies 0 β 1 1,..., β 1 β. Suppose that there exists i β 1 1,..., β 1 β but i k k 0 such that σ i [ Hββ Hβc ] > σ 0 [ Hββ Hβc ] or σ 0 [ Hββ Hβc ] > σ i [ Hββ Hβc ]. Then, since 0 < u β < e β and e β, u β = k k 0, for both cases, we always have 0 σ i [ Hββ Hβc ] u T β σ [ H ββ Hβc ] = 0 σ i [ Hββ Hβc ] 1 u β i β i= 0 1 u β i σ i [ Hββ Hβc ] > 0 σ 0 [ Hββ Hβc ] 1 u β i β i= 0 1 = σ 0 [ Hββ Hβc ] 0 k k 0 u β i u β i σ 0 [ Hββ Hβc ] β i= 0 1 u β i = 0, which implies that the second equality of 58 does not hold, which contradicts the assumption. Therefore, we know that H CX; θx if and only if ii holds. Case. S < k, i.e., e β, u β < k k 0. We know that β β 3 and k k 0 β 1 1,..., β. Suppose that 64 does not hold. Then, we know that either there exists i β 1 1,..., β such that i < k k 0 and σ i [ Hββ Hβc ] > σ 0 [ Hββ Hβc ] = 0 or σ 0 [ Hββ Hβc ] > 0. For the case that σ i [ Hββ Hβc ] > σ 0 [ Hββ Hβc ] = 0, since

16 16 Chao Ding 0 < u β < e β, we have 0 σ i [ Hββ Hβc ] u T β σ [ H ββ Hβc ] = 0 σ i [ Hββ Hβc ] 1 u β i β i= 0 1 u β i σ i [ Hββ Hβc ] > 0 σ 0 [ Hββ For the case that σ 0 [ Hββ Hβc ] 1 u β i β i= 0 1 u β i σ 0 [ Hββ Hβc ] = 0. Hβc ] > 0, since e β, u β < k k 0, we obtain that 0 σ i [ Hββ Hβc ] u T β σ [ H ββ Hβc ] = 0 σ i [ Hββ Hβc ] 1 u β i β i= 0 1 u β i σ i [ Hββ Hβc ] 0 σ 0 [ Hββ Hβc ] 1 u β i β i= 0 1 = σ 0 [ Hββ Hβc ] 0 k k 0 u β i u β i σ 0 [ Hββ Hβc ] β i= 0 1 u β i > 0. Therefore, for both cases, we always conclude that the second equality in 58 does not hold, which contradicts the assumption. Therefore, we know that H CX; θx if and only if iii holds. For the given S θx, let affcx; θx be the affine hull of the critical cone CX; θx, i.e., the smallest affine space containing CX; θx. Note that it follows from 56 that 0 CX; θx. It is easy to see cf. e.g., [33, Theorem.7] that affcx; θx = CX; θx CX; θx. Therefore, by Proposition 10, one can easily derive the following proposition on the characterization of affcx; θx. For simplicity, we omit the detail proof here. Proposition 11 Suppose that X, S R m n R m n is a solution of the GE 3. Let X = X S admit the SVD 7. Then, H affcx; θx if and only if H satisfies the following conditions: i if σ k X > 0, then there exists some τ R such that SU T SU T β 1 HV β1 0 0 β HV β = 0 τi β 0 ; 0 0 SU T β 3 HV β3 ii if σ k X = 0 and S = k, then there exists some τ R such that ] SU [U T T β HV β U T β 1 HV β β HV = 0 τi β 0 0 ; 0 0 U T b HV b U T b HV

17 Variational Analysis of the Ky Fan k-norm 17 iii if σ k X = 0 and S < k, then ] [U T β HV β U T β HV = SU T β 1 HV β Next, consider the dual GE 4. The critical cone of K at X, t = X S, θx 1 R m n R, associated with the complementarity problem in 5, is defined as C X, t; K = T K S, 1 X, θx. 65 Thus, we know from 39 that [ ] Ĥ, τ C X, t; K τik 0 Ĥ = H U V T, 0 0 H CX; θ S, 66 where CX; θ S R m n is defined by CX; θ S := H R m n ϑ S; H X, H = 0 if ϑs = 1, R m n if ϑs < We call CX; θ S the critical cone of θ S = N Bk S at X = X S, associated with the dual GE in 4. The following characterization of the critical cone CX; θ S can be obtain similarly as that of CX; θx. For simplicity, we omit the detail proof here. Proposition 1 Suppose that X, S R m n R m n is a solution of the dual GE 4. Assume that ϑs = 1. Let X = X S admit the SVD 7. Then, H CX; θ S ϑ S; H = X, H = 0, which is equivalent to the following conditions: i if σ k X > 0, then tru T β HV β = 0, [ ] SU T 0 0 α β 1 HV α β1 = 0 SU T β 1 HV β1 with SU T β 1 HV β1 0 and ] [ [U T β3 γhv β3 γ U T T ] SU β3 γhv = β 3 HV β with SU T β 3 HV β3 0; ii if σ k X = 0 and S < k, then [ ] SU T 0 0 α β 1 HV α β1 = 0 SU T β 1 HV β1 with SU T β 1 HV β1 0; iii if σ k X = 0 and S = k, then tru T β 1 β HV β1 β [ U T b HV b U T ] b HV 0, SU T α β 1 HV α β1 = [ ] SU T β 1 HV β1 with SU T β 1 HV β1 0. For the given X θ S, let affcx; θ S be the affine hull of the critical cone CX; θ S. Therefore, by Proposition 1, we obtain the following characterization of affcx; θ S.

18 18 Chao Ding Proposition 13 Suppose that X, S R m n R m n is a solution of the dual GE 4. Assume that ϑs = 1. Let X = X S admit the SVD 7. Then, H affcx; θ S if and only if H satisfies the following conditions. i If σ k > 0, then SU T α β 1 HV α β1 = [ ] SU T, tru T β HV β = β 1 HV β1 and ii If σ k = 0, then ] [ [U T β3 γhv β3 γ U T T SU β3 γhv = SU T α β 1 HV α β1 = β 3 HV β ]. 69 [ ] SU T. 70 β 1 HV β1 6 The second order analysis In this section, we shall study another important variational property of the Ky Fan k-norm θ = k, i.e., the conjugate function of the parabolic second order directional derivative of θ, which equals to the support function of the second order tangent set of the epigraph of θ. This conjugate function is closely related to the second order optimality conditions of the problem 1. For the given X, θx K, let T i, K X, θx; H, τ and T K X, θx; H, τ be the inner and outer second order tangent sets [, Definition 3.8] to K at X, θx K along the direction H, τ T K X, θx, respectively, i.e., T i, K X, θx ρh, τ K X, θx; H, τ := lim inf ρ 0 1 ρ and TK K X, θx ρh, τ X, θx; H, τ := lim sup 1, ρ 0 ρ where lim sup and lim inf are the Painlevé-Kuratowski outer and inner limit for sets cf. [34, Definition 4.1]. For TK := T i, K X, θx; H, τ or T K X, θx; H, τ, since K is convex, we know from [, Proposition 3.34, 3.6 & 3.63] that for any X, θx K and H, τ T K X, θx, TK T TK X,θX H, τ T K T TK X,θXH, τ, 71 where T TK X,θX H, τ is the tangent cone of T KX, θx at H, τ. For any given H, τ T K X, θx, let us consider the following two cases. Case 1. k σ ix; H = τ, i.e., H, τ bd T K X, θx. Since int K and the continuous convex function θ = k is parabolically second order directionally differentiable, we know from [, Proposition 3.30] that X, θx; H, τ = T K X, θx; H, τ = epi θ X; H,, T i, K where epi θ X; H, is the epigraph of the parabolic second order directional derivative of θ at X along the direction H, which is convex and given by epi θ X; H, := W, η R m n R k σ i X; H, W η. 7

19 Variational Analysis of the Ky Fan k-norm 19 Case. k σ ix; H < τ, i.e., H, τ int T K X, θx. Since T TK X,θXH, τ = R R m n, we know from 71 that T i, K X, θx; H, τ = T K X, θx; H, τ = R R m n. 73 Therefore, we may denote TK X, θx; H, τ the second order tangent set to K at X, θx along the direction H, τ T K X, θx. Next, we shall provide the explicit formula of the support function of the second order tangent set TK X, θx; H, τ. Let X, θx K be fixed. For any H, τ TK X, θx, denote T H, τ := TK X, θx; H, τ. Consider the support function δ T τ,h, : R Rm n, ], i.e., δt H,τS, ζ = sup S, W ζη W, η T H, τ, S, ζ R m n R. Claim δ T H,τ S, ζ if S, ζ / T TK X,θX H, τ. Proof. Let S, ζ / T TK X,θX H, τ be arbitrarily given. Since TTK X,θXH, τ is nonempty, we may assume that there exists W, η T TK X,θXH, τ such that S, ζ, W, η > 0. Fix any η, W T H, τ. By 71, we have for any ρ > 0, Therefore, we know that ρw, η W, η T TK X,θX H, τ T H, τ T H, τ. ρs, ζ, W, η S, ζ, W, η δ S, ζ T H, τ. Since S, ζ, W, η > 0 and ρ > 0 can be arbitrarily large, we conclude that δt H,τS, ζ for any S, ζ / T TK X,θX H, τ. Since K is a closed convex cone in R m n R, it can be verified easily that K T K X, θx T TK X,θXH, τ. In particular, we have ±X, θx T K X, θx T TK X,θX H, τ and ±H, τ T T K X,θXH, τ. Therefore, we know from the definition of the polar cone that if S, ζ T TK X,θX H, τ, then S, ζ K, S, ζ, X, θx = 0 and S, ζ, H, τ = Hence, by Claim 6, we only need to consider the point S, ζ R R m n satisfying the condition 74, since otherwise δt S, ζ. Moreover, instead of considering the general S Rm n, we only consider the point S such that X, S R m n R m n satisfying the GE 3, i.e., S θx, which is equivalent to the complementarity problem in 5. On the other hand, by the definition of the critical cone 53 of K, it is evident that the given point S, 1 satisfies the condition 74 if and only if H, τ CX, t; K with X, t = X, θx S, 1. Thus, by 54 and 56, we know that S, 1 satisfies the condition 74 if and only if H CX; θx defined by 55 and τ = S, H = k σ ix; H. Hence, we know from 7 that T H := T H, τ = W, η R m n R k σ i X; H, W η, 75

20 0 Chao Ding where for each i, the second order directional derivative σ i X, H, W is given by Proposition 4. Let X = X S admit the SVD 7. Let a 1,..., a r be the index sets defined by 16 with respect to X. Denote σ = σx and u = σs. Consider the following two cases. Case 1. σ k > 0. Let α, β and γ be the index sets defined by 31 and β 1, β and β 3 be the index sets defined by 35. Let r 0 r 1 0, 1,..., r 1, r 0 r 0 r 0 1 and r 1 1 r 1 r 1 be the integers such that 36 holds. For each l 1,..., r 0, since σ i = σ i for any i, i a l, we use ν l to denote the common value. By 54 and 56, we know that there exists an orthogonal matrix R O β such that 59 holds, i.e., Diagu β and SU T β HV β admit a simultaneous ordered eigenvalue decomposition. Therefore, R has the block diagonal structure 60. Hence, we know from the part i of Proposition 4 that W, η T H if and only if k σ i X; H, W = r 0 tr tr SU T r 0 a l W V al tr Ω al X, H R1 T T SU β W V β Ω β X, H R 1 0 i= β 1 1 λ i R T T SU β W V β Ω β X, H R η, 76 where Ω al X, H S m, l = 1,..., r 0 and Ω β X, H S m are given by with respect to X, R 1 OSU T β 1 HV β1 and R OSU T β HV β. Meanwhile, since u α = e α, we have for any W, η T H, η S, W = η U T SV, U T W V [ ] [ ] Diaguα 0 SU T = η, αw V α 0 0 Diagu β 0 SU T β W V β where r 0 = η tr SU T a l W V T a l Diagu β, SU T β W V T β r 0 = ΞW, η tr Ω al X, H Diagu β, Ω β X, H, 77 r 0 ΞW, η = η tr SU T a l W V T r 0 a l tr Ω al X, H Diagu β, SU T β W V T β Ω β X, H. 78 Next, we shall show that max ΞW, η W, η T H = In fact, since 0 u β e β and e β, u = k k 0, we know from Lemma 1 Fan s inequality that the last term of 78 satisfies tr Diagu β, SU T β W V T β Ω β X, H = R1 T T SU β W V β Ω β X, H R 1 tr R1 T T SU β W V β Ω β X, H R 1 Diagu β, R T SU T β W V T β Ω β X, H R T T SU β W V β Ω β X, H R u β, λ 0 i= β 1 1 R λ i R T T SU β W V β Ω β X, H R.

21 Variational Analysis of the Ky Fan k-norm 1 Therefore, together with 76 and 78, we obtain that for any W, η T H, ΞW, η 0. Also, it is easy to check that there exists W, η T H such ΞW, η = 0. By combining 77 and 79, we obtain that δt HS, 1 = sup S, W η W, η T H r 0 = tr Ω al X, H Diagu β, Ω β X, H. 80 Case. σ k = 0. Let α and β be the index sets defined by 34 and β 1, β and β 3 be the index sets defined by 35. Let r 0 0, 1,..., r 1 and r 0 r 0 r 0 1 be the integers such that 37 holds. For each l 1,..., r 0, since σ i = σ i for any i, i a l, we still use ν l to denote the common value. By 54 and 56, we know that there exist orthogonal matrices E O β and F O β n m such that 6 holds, i.e., [Diagu β 0] and [U T β HV β U T β HV ] admit a simultaneous ordered SVD, which implies that E and F have the block diagonal structure 63. Therefore, we know from the part ii and iii of Proposition 4 that W, η T H if and only if W, η satisfies the following conditions: if σ 0 [U T β HV β U T β HV ] > 0, then k σ i X; H, W r 0 = tr SU T a l W V T r 0 a l tr Ω al X, H tr S Q T 1 [U T β W H X HV β U T β W H X HV ]Q 1 0 i= β 1 1 if σ 0 [U T β HV β U T β HV ] = 0, then λ i S Q T [U T β W H X HV β U T β W H X HV ]Q η ; 81 k σ i X; H, W r 0 = tr SU T a l W V T r 0 a l tr Ω al X, H tr S Q T 1 [U T β W H X HV β U T β W H X HV ]Q 1 0 i= β 1 1 σ i Q T T [U β W H X HV β U T β W H X HV ]Q η, 8 where Ω al X, H S m, l = 1,..., r 0 are given by with respect to X, Q 1 O β1, Q O β, Q 3 O b and Q 3 O b n m are given by 63, Q O β β 3 and Q O β b n m are defined by [ ] [ ] Q Q 0 = and Q Q 0 = 0 Q 3 0 Q. 3

22 Chao Ding Meanwhile, since u α = e α, we have for any W, η T H, where η S, W = η U T SV, U T W V [ ] [ ] Diaguα 0 0 SU T = η, αw V α Diagu β 0 0 U T β W V β U T β W V r 0 = η tr SU T a l W V T a l [Diagu β 0], [U T β W V β U T β W V ] r 0 = ΞW, η tr Ω al X, H [Diagu β 0], [U T β H X HV β U T β H X HV ], 83 r 0 ΞW, η = η tr SU T a l W V T r 0 a l tr Ω al X, H [Diagu β 0], [U T β W H X HV β U T β W H X HV ]. 84 Similarly, we are able to show that max ΞW, η W, η T H = In fact, if σ 0 [U T β HV β U T β HV ] > 0, then since 0 u β e β and e β, u β k k 0, we know from Lemma 1 Fan s inequality that the last term of 84 satisfies [Diagu β 0], [U T β W H X HV β U T β W H X HV ] = [Diagu β 0], E T [U T β W H X HV β U T β W H X HV ]F tr S Q T 1 [U T β W H X HV β U T β W H X HV ]Q 1 0 i= β 1 1 λ i S Q T [U T β W H X HV β U T β W H X HV ]Q. Thus, together with 81 and 84, we obtain that ΞW, η 0 for any W, η T H. If σ 0 [U T β HV β U T β HV ] = 0, then by Lemma von Neumann s trace inequality, we know that [Diagu β 0], [U T β W H X HV β U T β W H X HV ] = [Diagu β 0], E T [U T β W H X HV β U T β W H X HV ]F tr S Q T 1 [U T β W H X HV β U T β W H X HV ]Q 1 0 i= β 1 1 σ i Q T T [U β W H X HV β U T β W H X HV ]Q.

23 Variational Analysis of the Ky Fan k-norm 3 Together with 8 and 84, we conclude that ΞW, η 0 for any W, η T H. Moreover, it is easy to check that in both case there exists W, η T H such that Ξη, W = 0 e.g., W = HX H R m n and η = k σ i X; H, W. By combining 83 and 85, we obtain that δt HS, 1 = sup S, W η W, η T r 0 = tr Ω al X, H Diagu β, U T β H X HV β. 86 We summarize the above results on the support function δt H set T H in the following proposition. of the second order tangent Proposition 14 Let X, S R m n R m n be a solution of the GE 3, i.e., S θx. Let X = X S admit the SVD 7. Denote σ = σx and u = σs. For any H CX, θx, let T H R m n R be the second order tangent set defined by 75, and Ω al X, H S m, l = 1,..., r 0 and Ω β X, H S m be the matrices given by with respect to X. Then, the support function of T H at S, 1 is given as follows. i If σ k > 0, then ii If σ k = 0, then r 0 δt HS, 1 = tr Ω al X, H Diagu β, Ω β X, H. r 0 δt HS, 1 = tr Ω al X, H Diagu β, U T β H X HV β. Remark By 75, we know that for the given S θx and H CX, θx, the second order tangent set T H is the epigraph of the closed convex function ψ := θ X; H, : R m n R. Then, the support function of T H at S, 1 obtained in Proposition 14 equals to the conjugate function value of ψ at S, i.e., ψ S := supw, S ψw W R m n = δ T HS, 1. Definition For any given X R m n, define the function Υ X : θx R m n R by for any S θx and H R m n, if σ k > 0, then if σ k = 0, then r 0 Υ X S, H := tr Ω al X, H Diagu β, Ω β X, H, r 0 Υ X S, H := tr Ω al X, H Diagu β, U T β H X HV β, where σ = σx, u = σs, and Ω al X, H S m, l = 1,..., r 0 and Ω β X, H S m are given by with respect to X.

24 4 Chao Ding Similarly, for the dual GE 4, by employing the similar arguments, we are able to derive the general results on the support function values corresponding to the second order tangent sets of the polar cone K. In particular, we are interesting in the support function value of the following the special second order tangent set T H at H CX; θ S, which is defined by T H := T K S, 1; H, 0 = epi ϑ S; H, if ϑs = 1, R m n R if ϑs < 1, where ϑ = k is the dual norm of the Ky Fan k-norm. For simplicity, we omit the detail proof here. Proposition 15 Let X, S R m n R m n be a solution of the dual GE 4. Suppose that X = X S has the SVD 7. Denote σ = σx and u = σs. For any H CX; θ S, let Ω α β1 S, H S α β1 and Ω al S, H S al, l = r 0 1,..., r 1 be the matrices defined by with respect to S. Then, the support function of T H at X, θx is given as follows. i If σ k > 0, then δ T H X, θx = r 0 ν l tr Ω α β1 S, H σ alal k tr Ω α β1 S, H β1β1 r 1 σ k l= r 0 1 tr Ω al S, H σ k tr U T β3 HS HV β3 Diagσ γ, U T γ HS HV γ. ii If σ k = 0, then δ T H X, θx = r 0 ν l tr Ω α β1 S, H. alal Definition 3 For any given S R m n, define the function Υ S : θ S R m n R by for any X θ S and H R m n, if σ k X > 0, then Υ S r 0 X, H := ν l tr Ω α β1 S, H σ alal k tr Ω α β1 S, H β1β1 r 1 σ k l= r 0 1 tr Ω al S, H σ k tr U T β3 HS HV β3 Diagσ γ, U T γ HS HV γ, if σ k X = 0, then Υ S r 0 X, H := ν l tr Ω α β1 S, H, alal where σ = σx, u = σs, and Ω α β1 S, H S α β 1 and Ω al S, H S a l, l = r 0 1,..., r 1 are given by with respect to S. It seems that the functions Υ X and Υ are quite complicate from the definitions. However, S one can easily compute the values by elementary calculations. Moreover, we have the following interesting proposition on the defined functions Υ X and Υ S.

25 Variational Analysis of the Ky Fan k-norm 5 Proposition 16 Let S θx or equivalently X θ S be given. Then, for any H R m n, Υ X S, H 0, Υ S X, H 0. Moreover, we have Υ X S, H = 0 Υ S X, H = 0, which is equivalent to the following conditions. i If σ k X > 0, then H αα Hαβ1 Hαβ H β1 α H β1 β Hβ1 1 β S α β 1 β, H β α H β β Hβ 1 β H β1 β 3 = H β3 β 1 T, Hβ β 3 = H β3 β T H αβ = H β α T = 0, Hαβ3 = H β3 α T = 0, H αγ = H γα T = 0, H β1 γ = H γβ1 T = 0, Hβ γ = H γβ T = 0, H αc = 0, Hβ1 c = 0, Hβ c = 0, 87 where H = U T HV, and the index sets α, β, γ, and β i, i = 1,, 3 are defined by 31 and 35. ii If σ k X = 0, then H αα S α, Hαβ1 = H β1 α T H αβ = H β α T = 0, Hαβ3 = H β3 α T = 0, 88 H αc = 0, where H = U T HV, and the index sets α, β, and β i, i = 1,, 3 are defined by 34 and 35. Proof. Let X = X S admit the SVD 7. Denote σ = σx and u = σs. Let H 1 = U T HV 1 and H = U T HV. Consider the following two cases. Case 1. σ k > 0. By and the definition of the pseudoinverse, we obtain that and tr Ω al X, H = l =1 l l S ν l ν H 1 al a l l l =1 1 ν l H al, l = 1,..., r 0 ν l ν l T H 1 al a l µ l tr Ω al X, H = r 0 l =1 µ l ν l σ k S H 1 al a l l =1 Thus, since u i = 0 if i β 3, we have l =r 1 1 µ l ν l σ k T H 1 al a l Diaguβ, Ω β X, H = r 1 l=r 0 1 µ l ν l σ k S H 1 al a l µ l σ k H al, l = r 0 1,..., r 1. µ l tr Ω al X, H.