Weaker hypotheses for the general projection algorithm with corrections

Transcription

1 DOI: /auom An. Şt. Univ. Ovidius Constanţa Vol. 23(3),2015, 9 16 Weaker hypotheses for the general projection algorithm with corrections Alexru Bobe, Aurelian Nicola, Constantin Popa Abstract In an earlier paper [J. of Appl. Math. Informatics, 29(3-4)(2011), ] we proposed a general projection-type algorithm with corrections proved its convergence under a set of special assumptions. In this paper we prove convergence of this algorithm under a much weaker set of assumptions. This new framework gives us the possibility to obtain as a particular case of our method the two-step algorithm analysed in [B I T, 38(2)(1998), ]. 1 Introduction We will consider the problem: find x IR n such that Ax b = min{ Az b, z IR n }, (1) where denotes the Euclidean norm (, will be the Euclidean scalar product). Concerning the matrix involved in (1) we will suppose throughout the paper that it has nonzero rows A i columns A j, i.e., A i 0, i = 1,..., m, A j 0, j = 1,..., n. (2) These assumptions are not essential restrictions of the generality of the problem (1) because, if A has null rows /or columns, it can be easily proved that Key Words: inconsistent least-squares problems, projection algorithms, correction 2010 Mathematics Subject Classification: AMS: 65F10, 65F20 Received: October, Revised: October, Accepted: October,

2 Weaker hypotheses for the general projection algorithm with corrections 10 they can be eliminated without affecting its set of classical or least squares solutions. We first introduce some notations: the spectrum spectral radius of a square matrix will be denoted by σ(b) ρ(b), respectively. By A T, N(A), R(A) we will denote the transpose, null space range of A. P S (x) will be the orthogonal (Euclidean) projection onto a vector subspace S of some IR q. S(A; b), LSS(A; b), x LS will st for the set of classical least squares solution of (1), respectively, the (unique) minimal norm solution. In the consistent case for (1) we have S(A; b) = LSS(A; b). In the general case the following properties are known (see [5], Chapter 1) x LS N(A), b = b A + b A, with b A = P R(A) (b), b A = P N(A T )(b), (3) LSS(A; b) = x LS + N(A) x LSS(A; b) Ax = b A, (4) S(A; b) = x LS + N(A) x S(A; b) Ax = b. (5) Let Q : n n R : n m be real matrices satisfying the (main) assumptions I Q = RA, (6) if Q = QP R(AT ), then Q < 1, (7) where Q denotes the spectral norm of the matrix Q. In the paper [4] we proposed the following algorithm (a stard example of such an algorithm is Kaczmarz s projection method; see Algorithm in [1]). Algorithm General Projections with Corrections. Initialization: x 0 IR n is arbitrary. Iterative step: x k+1 = Qx k + Rb + v k, (8) where We introduced in [4] the additional assumptions:, for ε k defined by v k R(A T ), k 0. (9) y IR m, Ry R(A T ), (10) ε k := v k + Rb A, (11) we assumed that there exist constants c > 0 δ [0, 1) such that Then, the following result was proved in [4]. ε k cδ k, k 0. (12)

3 Weaker hypotheses for the general projection algorithm with corrections 11 Theorem 1. Under the assumptions (6) (7) (9) (12), for any sequence (x k ) k 0 generated by the algorithm (8) we have P N(A) (x k ) = P N(A) (x 0 ), k 0 (13) lim k xk = P N(A) (x 0 ) + x LS LSS(A; b). (14) In this paper we will introduce a new set of assumptions weaker than (7) (12) we will prove that the algorithm (8) has the same convergence properties. Moreover, we will show that the two step algorithm presented by Elfving in [2] fits into the above mentioned more general hypotheses. 2 The new set of weaker assumptions We will keep the assumptions (6), (10) (9), but replace (7) with replace (12) with if Q = QP R(A T ), then ρ( Q) < 1. (15) By a direct application of (6) (10) we obtain lim k εk = 0. (16) if x N(A) then Qx = x N(A), (17) if x R(A T ) then Qx = x RAx R(A T ). (18) Moreover, the equalities (13) hold as in Lemma 1 of [4]. We can now prove the main result of our paper. Theorem 2. Under the hypotheses, I Q = RA, if Q = QP R(A T ), then ρ( Q) < 1, y IR m, Ry R(A T ), v k R(A T ), k 0, lim k εk = 0, any sequence (x k ) k 0 generated by the algorithm (8) converges (14) holds.

4 Weaker hypotheses for the general projection algorithm with corrections 12 Proof. According to (13) we can define the error vector of the iteration (8) as e k = x k (P N(A) (x 0 ) + x LS ). (19) Moreover, from (15) it follows that (I Q) is invertible (see, e.g., [3]) (I Q) 1 = I + Q + Q 2 + = j 0 Q j. (20) Then, as in [4] we obtain the equality from which we get e k+1 = Qe k + ε k, k 0, (21) e k = Q k e 0 + Q i ε i, k 1. (22) From (15) (see [3], Lemma ) there exist a matrix norm (depending on Q) such that Q < 1. (23) Let be a vector norm compatible with the above matrix norm, then Qx Q x, x IR n. (24) But, from (16) we obtain that lim k ε k = 0, thus (see also (23)) if ɛ > 0 is arbitrary fixed there exist an M > 0 an integer k 1 ɛ 1 such that Then we define k ɛ 1 by ε k M, k 1 (25) Q k ɛ, ε k < ɛ, k k 1 ɛ. (26) k ɛ = 2k 1 ɛ + 1 (27) consider k = k ɛ + µ, with an arbitrary integer µ 0. Now, we first take norms, use (26) split the sum in (22) as e k Q k e 0 + Q i ε i k 1 ɛ ɛ e 0 + Q i ε i + i=k 1 ɛ +1 Q i ε i. (28)

5 Weaker hypotheses for the general projection algorithm with corrections 13 For the first sum in (28), from (25) (26) we get k 1 ɛ Q i ε i M k 1 ɛ = M Q k1 ɛ Q ( i = M Q + + Q ) k1 ɛ ( Q ) k1 ɛ = M Q k1 ɛ +µ M Q k1 ɛ +µ 1 1 Q < 1 Q k1 ɛ +1 1 Q Mɛ 1 Q. (29) For the second sum in (28), from the above formula for k, (27) (26) we get Q i i=kɛ 1+1 ε i = Q k1 ɛ +µ 1 ε k1 ɛ +1 + Q k1 ɛ +µ 2 ε k1 ɛ Q ε k1 ɛ +µ 1 + ε k1 ɛ +µ ( ε k1 ɛ +1 Q ) k1 ɛ +µ ɛ +µ = ε k1 ɛ +1 1 Q k 1 Q 1 < ɛ 1 Q. (30) From (26) (28) (30) we conclude that, for an arbitrary ɛ > 0, there exist an integer k ɛ 1 (see also (27)), such that for any k k ɛ we have e k ɛ e 0 + Q i ε i ( ɛ e 0 Mɛ Q + ɛ 1 Q = ɛ e 0 + M + 1 ) 1 Q from which (14) follows the proof is complete. Corollary 1. Suppose that (6), (15), (16) hold, that (31) x k R(A T ), k 0. (32) Then, any sequence (x k ) k 0 generated by the algorithm (8) converges lim k xk = x LS. (33)

6 Weaker hypotheses for the general projection algorithm with corrections 14 Proof. We define the error vector e k by e k := x k x LS, k 0. (34) Because of (18) the fact that x LS R(A T ) we get thus, Qx LS = Qx LS, (35) (I Q)x LS = x LS QP R(AT )(x LS ) = x LS Qx LS = (I Q)x LS = RAx LS = Rb A, Rb A = (I Q)x LS = (I Q)x LS. (36) Consequently, by using (34), (8), (32) (36) we successively get, k 0 e k+1 = x k+1 x LS = Qx k + Rb + v k x LS = Qx k +Rb+v k ( Qx LS +Rb A ) = Q(x k x LS )+v k +Rb A = Qe k +ε k. (37) Then, as in the proof of Theorem 2 we obtain (22) for e k from (34) then (33), because for x 0 R(A T ) we have P N(A) (x 0 ) = 0. This completes the proof. Next we show that the two steps algorithm of Elfving [2] is a special case of our algorithm. Algorithm Elfving (ELF). Initialization: Iterative step: x 0 R(A T ), y 0 = b Az 0, for some z 0 IR n. (38) y k+1 = (I AΓ)y k, (39) x k+1 = Qx k + R(b y k+1 ). (40) The matrices Q : n n, R : n m, Γ : m n satisfy Q + RA = I, (41) for w R(A), z = Qz + Rw if only if Az = w, (42) ρ((i AΓ)P R(A) ) < 1, (43) ρ(qp R(AT )) < 1, (44) z N(A T ) = Γz = 0, (45) u R(A) = Ru R(A T ). (46)

7 Weaker hypotheses for the general projection algorithm with corrections 15 Proposition 1. (i) The algorithm (ELF) is identical with algorithm (8), with the corrections v k defined by v k := Ry k+1. (47) (ii) The assumptions (6), (15), (16) (32) are satisfied. Proof. The assumptions (6) (15) are identical with (41) (44), respectively. To show that assumption (32) holds we argue by mathematical induction, as in the proof of Proposition 3 in [4]. Assumption (16) holds because, from (47), again as in the proof cited before, we get v k + Rb A = Ry k+1 + Rb A = R [ (I AΓ)P R(A) ] k+1 (ba ). According to the hypothesis (43) Theorem 1, from Chapter 1 in [5], the matrix (I AΓ)P R(A) is convergent, i.e. lim k ((I AΓ)P R(A) ) k = 0, from which we get (16) the proof is complete. Acknowledgements. The authors wish to thank to the anonymous referees for their valuable comments that improved the initial version of the paper. References [1] Censor Y., S. A. Zenios Parallel Optimization: Theory, Algorithms Applications, Oxford Univ. Press, New York, [2] Elfving T., A stationary iterative pseudoinverse algorithm, B I T, 38(2) (1998), [3] Horn R. A., Johnson C. R., Matrix Analysis, Cambridge University Press, New York, [4] Nicola A., Popa C. Rüde U., Projection algorithms with correction, Journal of Applied Mathematics Informatics, 29 (3-4) (2011), [5] Popa C., Projection Algorithms - Classical Results Developments. Applications to Image Reconstruction, Lambert Academic Publishing - AV Akademikerverlag GmbH & Co. KG, Saarbrücken, Germany, 2012.

8 Weaker hypotheses for the general projection algorithm with corrections 16 Alexru Bobe, Ovidius University of Constanta, Faculty of Mathematics Informatics, Blvd. Mamaia 124, , Constanta, Romania Aurelian Nicola, Ovidius University of Constanta, Faculty of Mathematics Informatics, Blvd. Mamaia 124, , Constanta, Romania Constantin Popa, Ovidius University of Constanta, Faculty of Mathematics Informatics, Blvd. Mamaia 124, , Constanta, Romania