A null space method for solving system of equations q

Size: px

Start display at page:

Download "A null space method for solving system of equations q"

Terence Beasley
5 years ago
Views:

1 Applied Mathematics and Computation 149 (004) A null space method for solving system of equations q Pu-yan Nie 1 Department of Mathematics, Jinan University, Guangzhou 51063, PR China Abstract We transform the system of nonlinear equations into a nonlinear programming problem, which is solved by null space algorithms. We do not use standard least square approach. We divide the equations into two groups. One group contains the equations that are treated as equality constraints. The square of other equations is regarded as objective function. Two groups are updated in every step. In essence, two different methods are used for a system of equations in an algorithm. Ó 003 Elsevier Inc. All rights reserved. Keywords: Null space methods; Equality constraints; Nonlinear system of equations; Nonlinear programming; Global convergence 1. Introduction In this paper, we consider the system of nonlinear equations of the following form c i ðxþ ¼0 i ¼ 1; ;...; m; ð1:1þ where x R n. When (1.1) is solved by iterative methods, we use x k ; k ¼ 1; ;... to denote the successive iterates. An important method is based on successive q Supported partially by Chinese NNSF grants and the knowledge innovation program of CAS. address: pynie00@hotmail.com (P.-y. Nie). 1 State Key Laboratory of Scientific and Engineering Computing, Institute of Computational Mathematics and Scientific/Engineering Computing, Academy of Mathematics and System Sciences, Chinese Academy of Sciences, P.O. Box 719, Beijing , PR China /$ - see front matter Ó 003 Elsevier Inc. All rights reserved. doi: /s (03)

2 16 P.-y. Nie / Appl. Math. Comput. 149 (004) 15 6 linearization, in which d k is calculated on iteration k that solves the system of linear equations c ðkþ i þða ðkþ i Þ T d ¼ 0 i ¼ 1; ;...; m; ð1:þ where c ðkþ i ¼ c i ðx k Þ; a ðkþ i ¼frc i ðx k Þg ði ¼ 1; ;...; mþ. Ifm ¼ n (1.) is NewtonÕs method. NewtonÕs method has local second order convergence near a regular solution. But in some cases (1.) is inconsistent. An alternative approach is to pose (1.1) as a minimization problem minimize hðxþ ¼cðxÞ T cðxþ: ð1:3þ This problem can also be solved by successive linearization. This idea helps to improve the global properties of NewtonÕs method. But there are still potential difficulties. Powell [7] gives an example that the iterates converges to a nonstationary point of hðxþ, which is obviously unsatisfactory. Therefore, we consider to build a constrained optimization problem to solve (1.1). Namely, we divide the set f1; ;...; mg into two parts S 1 and S, where S denotes the complement f1; ;...; mg=s 1. Thus, our problem becomes X minimize c i ðxþ is 1 ð1:4þ subject to c j ðxþ ¼0; j S : The choice of S 1 and S will be given behind our algorithm in Section of this paper. When (1.1) is infeasible, a local minimization of hðxþ > 0 is found or a point is located at which the linearized system (1.) is infeasible, which is regarded as local infeasibility. Just as the global minimization of hðxþ, it is very difficult to describe global infeasibility. There are rare paper devoting to using optimization method to solve nonlinear system of equations. In this paper, we try to use optimal methods to work out (1.1). The paper is organized as follows: In Section, an algorithm is put forward. The algorithm is analyzed in Section 3. Some numerical results and remarks are listed in Section 4.. A null space algorithm There exist extensive research about null space methods [,4,6,9,10] for nonlinear programming. But we hope to attack nonlinear system of equations with null space methods in this work. In other words, we aim to solve (1.1) based on (1.4). At the beginning of the kth iteration, we hope to find a better point. In a null space technique, a step is consisted of two component, namely, a null space step and a range space step. The null space step (or normal) is obtained by solving the subproblem

3 P.-y. Nie / Appl. Math. Comput. 149 (004) minimize subject to ka T S h þ c S ðx k Þk khk 6 nd; ð:1þ where n ð0; 1Þ is a relaxation parameter, c S ðxþ is the vector of c j ðxþ for j S and A S ¼rc S ðx k Þ T. The range (vertical) step is based on the following subproblem minimize W k ðh k þ Z k vþ¼ 1 ðz kv þ h k Þ T B k ðz k v þ h k Þþg T k ðz kv þ h k Þ subject to kvk 6 D; ð:þ where A k Z k ¼ 0 and Zk TZ k ¼ I, B k is Hessian or approximate Hessian matrix, ðg k Þ i ¼ o P is 1 c i ðxþ=ox i and h k is the solution of (.1). In (.1) and (.) -norm or some other norm is used. We define the new point as x k ðdþ ¼x k þ Z k v k þ h k : When a new point is accepted, the sets S 1 and S are updated by some strategy. Then, to solve (1.1) we work out (1.4) using null space method. The step is obtained by (.1) and (.). In order to decide whether to use or deny the trial point, some criterion is used. Thus, the following merit function is used, which is similar to that in [5]. m k ðxþ ¼ X is 1 c i ðxþ þ r k kc S ðxþk; ð:3þ where r k > 0 is a parameter. Therefore, we define the predicted reduction and actual reduction to judge the new trial point as follows. aredðs k ; r k Þ¼ ðm k ðx k þ s k Þ m k ðx k ÞÞ; ð:4þ predðs k ; r k Þ¼ 1 st k B ks k þ g T k s k þ r k ka T S s k þ c S ðx k Þk r k kc S ðx k Þk : ð:5þ Algorithm 1 (Null space algorithm for system of equations) Step 0. Choose D 0 ; x 0 ; g;;c; n; S k 1 ; Sk where D 0 > 0; 0 < g < 1; D max P D 0, compute g 0 ; k 0 ; c i ðx 0 Þ; A k, i S k. r 0 ¼ r 1 P 1; ^r > 0. Set k :¼ 0. Step 1. If kc k k 6 then stop. Otherwise compute Z k. Step. If c i ðx k Þ¼0; i S. Then go to Step 3. Or else, compute (.1) to get h k. Step 3. Compute (.) to get v k.ifv k ¼ 0 and h k ¼ 0 then stop. Step 4. Set s k ¼ h k þ Z k v k : Compute the actual reduction aredðd k Þ and the predicted reduction predðd k ; r k 1 Þ.

4 18 P.-y. Nie / Appl. Math. Comput. 149 (004) 15 6 Step 5. If predðs k ; r k 1 Þ P ðr k 1 =Þðkc S k kc S þ A S s k kþ; then set r k ¼ r k 1. Otherwise set r k ¼ s T k B ks k þ g T k s k ðkc S k kc S þ A S s k kþ þ ^r: ð:6þ Step 6. If r ¼ aredðs k ; r k Þ=predðs k ; r k Þ P g then set x kþ1 ¼ x k þ s k and D max P D kþ1 P D k ; or else set x kþ1 ¼ x k and D kþ1 6 D k. Step 7. Compute g kþ1 ; B kþ1 ; A kþ1 and S kþ1 1 ; S kþ1. Let k ¼ k þ 1 and go to Step 1. In the above algorithm, A k Z k ¼ 0andZ T k Z k ¼ I. For the sets S k 1 and Sk,we can define them from null space technique. In a null space algorithm, Yuan [11] points out that: the null space step is the NewtonÕs step while the vertical step is a quasi-newtonõs step if we use some kind of quasi-newton method to update B k. Thus, there will appear one-fast one-slow pattern. Yuan [11] uses two vertical components and one null space component as a step. Now we discuss the sets S 1 and S. For convenience, we denote jc i1 j 6 jc i j 6 6 jc im j; ð:7þ where fi 1 ; i ;...; i m g is a permutation of f1; ;...; mg. In this work, we define S 1 ¼fi tþ1 ;...; i m g; S ¼fi 1 ; i ;...; i t g; ð:8þ ð:9þ where 1 6 t < m is a constant integer. we choose S 1 and S in such way that the second information of some equations with large residual are made full use of. Certainly, there are other choices of S 1 and S. For example, S 1 ¼fi 1 ; i ;...; i t g; S ¼fi tþ1 ;...; i m g: ð:10þ ð:11þ But we do not analyze the properties under (.10) and (.11). Hence, the results in Section 3 is based on (.8) and (.9). There are some advantages about optimization techniques to solve system of equations. Firstly, it provides another way of getting a local infeasibility point. When a point lies near a local infeasibility point, it may be very slow to find a local infeasibility point. But it can be found immediately with optimization method because some second information is facilitated. Secondly, when the equations have different properties to a great degree, it will help to balance them to a certain degree. Thirdly, this is a method without Lagrangian multiplier and sometime Lagrangian multiplier is problematic. Finally, because there are two methods for null space step and range step, it is the first time to use two different methods to solve (1.1) in essence.

5 P.-y. Nie / Appl. Math. Comput. 149 (004) Convergence properties of null space algorithm Null space method has global convergence property and local superlinear convergence. We hope that there are some good results for our algorithm. In order to give the global convergence we make some assumptions as follows, which is called standard assumptions. Assumption 1 (1) fx k gx and X is nonempty and bounded; () c i ðxþ ði ¼ 1; ;...; mþ are twice continuously differentiable on an open set containing X ; (3) the matrix sequence fb k g is bounded. For convenience, we define the local infeasibility point of (1.1). Definition 1. If x H is a local minimization of (1.4) but not the solution of (1.1), we call x H a local infeasibility point of (1.1). Then, we analyze the properties to Algorithm 1 based on the above assumptions. When the algorithm terminates finitely, a solution of (1.1) or a local infeasibility point is obtained. It is apparent that the following result is true. Theorem 1. If algorithm terminates at Step 1, then a solution to (1.1) is found. If the algorithm terminates at Step 3, then a local infeasibility point is obtained. Proof. The first part is obvious. When the algorithm terminates at Step 3, the final iterate point is not a solution of (1.1) because it does not satisfy the condition of Step 1. But it satisfies the KKT conditions. Therefore, it is a local minimization of (1.4). Hence, it is a local infeasibility point. Of course, in our algorithm there are chances to terminate at Step 3 because the choice of S 1 and S. But if we give an alternative choice about S 1 and S there is little possibility. When the algorithm terminates infinitely, we investigate the point sequence. Certainly, we assume that the solutions to (.1) and (.) satisfy certain descent conditions. Assumption (1) A T S has full column rank, Z k is bounded for all k, A þ S is uniformly bounded. () The solution to (.1) and (.) satisfy Cauchy descent condition, namely, kc S k ka S h k þ c S k P s 1 kc S k minfkc S k; nd k g; ð3:1þ

6 0 P.-y. Nie / Appl. Math. Comput. 149 (004) 15 6 W k ðh k Þ W k ðh k þ Z k v k Þ P s kz k g k k minfs 3 kz k g k k; D k g; ð3:þ where s 1 ; s ; s 3 are positive constants independent of k. (3) kh k k 6 s 4 kc S k, where s 4 is a positive constant independent of k. () is a weaker condition because Cauchy point satisfies it. Further, there are many algorithms satisfy (3) under (1) (see [3]). Hence, the above assumptions are rational. Then, we estimate the predicted reduction to get some results about our algorithm. Lemma 1. The sequence r k and predicted reduction satisfy r k P r k 1 P 1; predðs k ; r k Þ P r k ðkc S k kc S þ A S s k kþ; ð3:3þ W k ð0þ W k ðh k Þ P s 5 minfkc S k; nd k g; ð3:4þ where s 5 is a positive constant independent of k. Proof. The first inequality and (3.3) is obtained from Algorithm 1 directly. Then, we show (3.4). From Assumption we have W k ð0þ W k ðh k Þ¼ g T k h k h T k B kh k ¼ ðg T k þ ht k B kþh k P kg T k þ ht k B kkkh k k P s 6 kh k k P s 6 s 4 minfkc S k; nd k g ¼ s 5 minfkc S k; nd k g; where s 6 is a positive constant independent of k, which is the result. It is apparent that r k is monotonic increasing sequence. Then, we show that r k is a bounded sequence. Lemma. fr k g is a sequence generated by our algorithm. Then, r k ¼ r for sufficiently large k. Proof. If predðs k ; r k 1 Þ P ðr k 1 =Þðkc S k kc S þ A T S s k kþ, we have r k ¼ r k 1. Otherwise, by definition (.5) we have r k 1 ðkc S k kc S þ A S s k kþ 6 1 st k B ks k þ g T s k ¼ W k ðs k Þ W k ð0þ 6 W k ðh k Þ W k ð0þ 6 s 5 minfkc S k; nd k g: ð3:5þ

7 P.-y. Nie / Appl. Math. Comput. 149 (004) Inequalities (3.5) and (3.1) show that r k 1 6 4s 5 =s 1. Thus, we see that r k is bounded above. Consequently, r k ¼ r for sufficiently large k. Then the convergence result is listed as follows. Theorem. Under Assumptions 1 and, assume that kk is -norm and r P kc S kðx kþ1 Þk þ kc S kþ1ðx kþ1 Þk, then we have lim inf½kc S kþkz k g k k Š¼0: ð3:6þ k!1 Namely, it has an accumulation which is the solution of (1.1) or a local infeasibility point. Proof. If the result were false there should have been 0 > 0 such that kc S kþkz k g k k > 0 ; ð3:7þ for all k. We should prove the following results: (1) m k ðx k Þ is bounded for all k. () There exists k P k 0 such that m k ðx kþ1 Þ P m kþ1 ðx kþ1 Þ and r k ¼ r. (3) predðs k ; r k Þ P s 8 0 minf 0 ; D k g for all k. (1) is obvious because of Assumption 1 and Lemma. Then, we show (). It is obvious that there exists k P k 0 such that r k ¼ r from Lemma. Actually, m k ðx kþ1 Þ¼ P m j¼1 c iðx kþ1 Þ þ rkc S kðx kþ1 Þk P js c k j ðx kþ1 Þ : While m kþ1 ðx kþ1 Þ¼ P m j¼1 c iðx kþ1 Þ þ rkc S kþ1ðx kþ1 Þk P js c kþ1 j ðx kþ1 Þ. According to the definition of S we have kc S kþ1ðx kþ1 Þk 6 kc S kðx kþ1 Þk. Thus, m k ðx kþ1 Þ m kþ1 ðx kþ1 Þ¼rðkc S k þkc S kþ1 ¼ðr ðkc S k ðx kþ1 Þk kc S kþ1 ðx kþ1 ÞkÞðkc S k kc S kþ1ðx kþ1 ÞkÞ P 0: ðx kþ1 Þk þ kc S kþ1 ðx kþ1 ÞkÞ ðkc S kðx kþ1 Þk ðx kþ1 ÞkÞ ðx kþ1 Þk kc S kþ1 ðx kþ1 ÞkÞÞðkc S k ðx kþ1 Þk Hence, () is true, which means that there are only finitely many iterates which violate m k ðx kþ1 Þ P m kþ1 ðx kþ1 Þ. From (3.1) and (3.3) we have predðs k ; rþ P r 4 s 1 minfkc S k; nd k g: ð3:8þ On the other hand,

8 P.-y. Nie / Appl. Math. Comput. 149 (004) 15 6 predðs k ; rþ ¼Wð0Þ Wðs k Þþrðkc S ðx k Þk kc S ðx k ÞþA S s k kþ ¼ Wð0Þ Wðh k ÞþWðh k Þ Wðs k Þ þ rðkc S ðx k Þk kc S ðx k ÞþA S s k kþ P s 5 minfkc S k; nd k gþs kz k g k k minfs 3 kz k g k k; Dg: ð3:9þ Multiples the above inequalities (3.8) and (3.9) by 8s 5 =ð8s 5 þ rs 1 Þ and rs 1 =ð8s 5 þ rs 1 Þ respectively and the sum of the inequalities we obtain predðs k ; rþ P rs 1s 5 minfkc S k; nd k g 8s 5 þ rs 1 þ rs 1 s kz k g k k minfs 3 kz k g k k; Dg: 8s 5 þ rs 1 Thus, (3) is obtained from the above inequality. Then, we show our result about this theorem. For convenience we denote I ¼fkjkPk 0 and r ¼ aredðs k ;r k Þ=predðs k ;r k ÞPgg. Because of above (1) and () we have 1 > X1 ðm k ðx k Þ m k ðx kþ1 ÞÞ P X ðm k ðx k Þ m k ðx kþ1 ÞÞ k¼1 ki P X g predðs k ; r k Þ P X gs 8 0 minf 0 ; D k g: ki ki ð3:10þ Thus, lim k!1 D k ¼ 0. On the other hand, when D k is small enough, aredðs k ; r k Þ¼predðs k ; r k ÞþOðD k Þ: Thus, r ¼ aredðs k; r k Þ=predðs k ; r k Þ P g. Then, D kþ1 P D k, which is contradicted with lim k!1 D k ¼ 0. Therefore, (3.7) is not true and our result is established. In fact, r P kc S k ðx kþ1 Þk þ kc S kþ1ðx kþ1 Þk is a moderate condition, which is very easy to satisfy. We have given the global convergence to Algorithm 1. We do not consider which kind of step but require Cauchy descent condition (3.1) and (3.). In some cases, we get the solution to (1.1). But in other cases, (1.1) has no solutions globally or locally. Now we give a condition that the KKT points to our problem is just the solution to (1.1). Theorem 3. If (1.1) has a solution and the gradients of c i ðxþ are independent, then, a KKT point is just the global minimum. Proof. If x H is a KKT point to (1.4), then X X c i ðx H Þ5 j c i ðx H Þþ k i 5 j c i ðx H Þ¼0; is 1 is for j ¼ 1; ;...; n. Because the gradients of c i ðx H Þ are independent, c i ðx H Þ¼0 for i S 1. Meanwhile, c S ¼ 0 is obvious. Hence, our result holds.

9 It is very difficult to analyze the cases which 5c is dependent. But in some special cases, our algorithm can avoid the bad result. For example, if c i ðxþ 6¼ 0; c j ðxþ 6¼ 0andrc i ; rc j are dependent, the subproblem based on (1.) is inconsistent. But if c i and c j belong S 1 and S respectively, we can find a new point to avoid this bad situation. Now, we analyze its local properties about Algorithm 1. Just as other analysis about local properties, we require the exact solution to (.1) and (.). Firstly, we make some assumptions as follows. Assumption 3 (1) fx k g!x H, where x H is a KKT point, namely c S ðx H Þ¼0 and there exists k H R k such that gðx H Þ Aðx H Þk H ¼ 0: P.-y. Nie / Appl. Math. Comput. 149 (004) ð3:11þ () Aðx H Þ is full column rank; (3) for any d k ¼ Z k v k R n, k 1 dk Td k 6 dk TB kd k 6 k dk Td k where 0 < k 1 6 k are constants and kzk T lim ðb k W H ÞZ k Zk Ts kk ¼ 0; ð3:1þ k!1 ks k k P where ðz H Þ T W H Z H is positive definite and W H ¼r f ðx H Þ k i¼1 ðkh Þ i r c S ðx H Þ, which is the Hessian of Lagrange function. We assume that ks k k!0 from Assumption 3 to our algorithm. Similar to Powell and YuanÕs results [8] we have Theorem 4. There exists an integer k 0 such that D k P ^D > 0; for all k > k 0. ð3:13þ Proof. When ks k k is small enough, from Taylor expansion we have predðs k ; r k Þ¼aredðs k ; r k ÞþOðks k k Þ: ð3:14þ When kh k k¼oðkz k v k kþ or kz k v k k¼oðkh k kþ then predðs k ; r k Þ¼Oðkh k kþ ¼ Oðks k kþ. Therefore, predðs k ; r k Þ=aredðs k ; r k Þ P g when ks k k is small enough. When kh k k¼oðkz k v k kþ, ks k k¼kz k v k þ h k k P k 3 kz k v k k where k 3 > 0 is a constant from Assumption 3. Then, predðs k ; r k Þ¼OðWðh k Þ Wðh k þ Z k v k ÞÞ ¼ OðkZ k v k k Þ: Therefore, predðs k ; r k Þ=aredðs k ; r k Þ P g when ks k k is small enough. Thus, in the both cases, D will not be reduced if ks k k is small enough, which is our results.

10 4 P.-y. Nie / Appl. Math. Comput. 149 (004) 15 6 Theorem 5. Under Assumptions 1 3, assume that S k 1 is not changed when k > k 0. If (3.1) is true, then kx kþ1 x H k lim k!1 kx k 1 x H k ¼ 0: ð3:15þ If kzk T lim ðb k W H Þs k k ¼ 0; k!1 ks k k ð3:16þ then kx kþ1 x H k lim k!1 kx k x H k ¼ 0: ð3:17þ The results is a direct result from null space algorithm. Thus, the locally superlinear convergence result is obtained, which is closed related to the update of Hessian matrix. The assumption, which S 1 and S are unchanged, is reasonable to a certain degree because of the choice of S 1 and S. Meanwhile, in the null space the convergent rate is lower than that in vertical space locally. 4. Conclusion remarks and numerical results We give a null space method to solve (1.1). For the system of nonlinear equations, two different methods are used in an algorithm to work out it. Our analyses are based on the partition (.8) and (.9). We let the number of constraints is constant. An interesting question is how to choose t. Certainly, we can judge from the initial point and initial values. For the extreme cases if the number of constraints is m, our algorithm is a NewtonÕs method to work out (1.1). If the number of constraints to be 0, our algorithm is least square approach. In fact, we know that the convergence result is true if m k ðx kþ1 Þ P m kþ1 ðx kþ1 Þ; ð4:1þ for r k ¼ r. If we change the number of constraints in some ways, the same results will be achieved because (4.1) is satisfied. For example, S 1 ¼fi tk þ1;...; i m g; S ¼fi 1 ; i ;...; i tk g; ð4:þ ð4:3þ and m P t k P t kþ1 P 1. We can use the other kinds of norm, such as 1-norm and 1-norm. Then the problem becomes

11 P.-y. Nie / Appl. Math. Comput. 149 (004) minimize kc S1 ðxþk ð4:4þ subject to c j ðxþ ¼0; j S : If S 1 and S are based on (.8) and (.9), the same results as those in Section 3 can be obtained when r P kc S kðx kþ1 Þk þ kc S kþ1ðx kþ1 Þk. For B k, we can use one-side reduced Hessian update or two-side reduced Hessian update. It is an important issue. But we do not discuss it in detail. Our algorithm is very flexible because we just require Assumptions 1 and to be satisfied. We do not limit to some special algorithm step. For choice of initial trust region radius and update, no specific method is given. Now, we give examples to our algorithm. The first example comes from [7], which converges to a nonstationary point if least square approach is used, and t ¼ 1. Example 1. Consider the problem of finding a solution of nonlinear system x F ðx; yþ ¼ 10x=ðx þ 0:1Þþy ¼ 0 : ð4:5þ 0 The unique solution is ðx H ; y H Þ¼ð0; 0Þ. It has been proved in [7] that, starting from ðx 0 ; y 0 Þ¼ð3; 1Þ, the iterates generated by the least square algorithm converge to z ¼ð1:8016; 0:0000Þ, which is not a stationary point. But using our algorithm, we can get a sequence of points converge to ðx H ; y H Þ. Our is 1.0e 6. The iterate number is 10. The final residual is kf k¼5:5134e 7. The following example appears in [1], which converges to a nonstationary point if NewtonÕs method is used, and t ¼ 1. Example. Consider the problem of finding a solution of nonlinear system x þ y F ðx; yþ ¼ ¼ 0 : ð4:6þ ðx 1Þy 0 The unique solution is ðx H ; y H Þ¼ð0; 0Þ. Let us define the line C ¼fð1; yþ : y Rg: If the starting point ðx 0 ; y 0 ÞC. Then the Newton method for (1.) are confined to C (see [1]). Meanwhile, if we choose S 1 which based on (.8) and (.9), the same situation as that in [1]. But if we choose S 1 and S with (.10) and (.11) initially and then we use (.8) and (.9), the optimal result can be obtained. The number of calculation to function is 8 and the number of calculation to gradient is 6 if the initial point is (1.0,.0). But if we choose the initial point as (1.0, 0.0), the methods based on (1.1) and (1.) will be failed. But if the above technique is used, the optimal solution will be obtained with NF ¼ 4; NG ¼. Certainly, If there are finitely many steps based on (.10) and

12 6 P.-y. Nie / Appl. Math. Comput. 149 (004) 15 6 (.11), the convergent results are obtained with the same methods as our results. But it is interesting to investigate our algorithm when S 1 and S are given by (.10) and (.11). Acknowledgement The author thanks Prof. Yuan Ya-xiang for his beneficial advice which improve this paper to a great degree. References [1] R.H. Byrd, M. Marazzi, J. Nocedal, On the convergence of Newton iterations to nonstationary points. Report OTC 001/01, Optimization Technology Center, Northwestern University, Evanston, Argonne, IL 6008, USA, 001. [] J.E. Dennis Jr., M. El-Alem, M.C. Maciel, A global convergence theory for general trust region based algorithms for equality constrained optimization, SIAM J. Opt. 7 (1997) [3] J.E. Dennis Jr., M. Heinkenschloss, L.U. Vicente, Trust region interior-point SQP algorithms for a class of nonlinear programming problems, SIAM J. Control Opt. 36 (1998) [4] J.E. Dennis Jr., R.B. Schnable, Numerical Methods for Unconstrained Optimization and Nonlinear Equations, Prentice-Hall, Englewood Cliffs, NJ, [5] R. Fletcher, in: Practical Methods of Optimization, Constrained Optimization, Vol., John Wiley and Sons, New York and Toronto, [6] M. Lalee, J. Nocedal, T.D. Plantenga, On the implementation of an algorithm for large-scale equality constrained optimization, SIAM J. Opt. 8 (3) (1998) [7] M.J.D. Powell, A hybrid method for nonlinear equations, in: P. Rabinowitz (Ed.), Numerical Methods for Nonlinear Algebric Equations, Gordon and Breach, London, [8] M.J.D. Powell, Y. Yuan, A recursive quadratic programming algorithm for equality constrained optimization, Math. Prog. 35 (1986) [9] A. Vardi, A trust region algorithm for equality constrained minimization: convergence and implementation, SIAM J. Numer. Anal. (3) (1985) [10] Y. Yuan, An only -step Q-superlinear convergence example for some algorithms that use reduced Hessian approximation, Math. Prog. 3 (1985) [11] Y. Yuan, A null space algorithm for constrained optimization, in: Z.C. Shi et al. (Eds.), Advances in Scientific Computing, Science Press, Beijing, 001, pp

An interior point type QP-free algorithm with superlinear convergence for inequality constrained optimization

Applied Mathematical Modelling 31 (2007) 1201 1212 www.elsevier.com/locate/apm An interior point type QP-free algorithm with superlinear convergence for inequality constrained optimization Zhibin Zhu *