Conjugate Gradient Projection Approach for MIMO Gaussian Broadcast Channels

Transcription

1 Conjugate Gradent Projecton Approach for MIMO Gaussan Broadcast Channels Ja Lu Y. Thomas Hou Sastry Kompella Hanf D. Sheral Department of Electrcal and Computer Engneerng, Vrgna Tech, Blacksburg, VA 4061 Informaton Technology Dvson, U.S. Naval Research Laboratory, Washngton, DC 0375 Department of Industral and Systems Engneerng, Vrgna Tech, Blacksburg, VA 4061 Abstract Researchers have recently shown that the drtypaper codng (DPC) s the optmal transmsson strategy for multple-nput multple-output Gaussan broadcast channels (MIMO BC). Moreover, by the channel dualty, the nonconvex MIMO BC sum rate problem can be transformed to the convex dual MIMO multple-access channel (MIMO MAC) problem wth a sum power constrant. In ths paper, we desgn an effcent algorthm based on conjugate gradent projecton (CGP) to solve the MIMO BC maxmum sum rate problem. Our proposed CGP algorthm solves the dual sum power MAC problem by utlzng the powerful concept of Hessan conjugate. We also develop a rgorous algorthm to solve the projecton problem. We show that CGP enjoys provable convergence, scalablty, and effcency for large MIMO BC systems. I. INTRODUCTION Recently, there s great nterest n characterzng the capacty regon for multple-nput multple-output (MIMO) broadcast channels (MIMO BC) and MIMO multple-access channels (MIMO MAC). Most notably, Wegarten et. al. [1] proved the long-open conjecture that the drty paper codng (DPC) strategy s the capacty achevng transmsson strategy for MIMO BC. Moreover, by the channel dualty between MIMO BC and MIMO MAC establshed n [] [4], t can be shown that the nonconvex MIMO BC sum rate problem can be transformed to the convex dual MIMO MAC problem wth a sum power constrant. However, although the standard nteror pont convex optmzaton method can be used to solve the sum power MIMO MAC problem, ts complexty s consderably hgher than those methods that explot the specal structure of the sum power MIMO MAC problem. Such custom desgned algorthms nclude the mnmax method (MM) [5], the steepest descent (SD) method [6], the dual decomposton (DD) method [7], and two teratve water-fllng methods (IWFs) [8]. Among these algorthms, MM does not have lnear complexty and s more complex than the others. SD and DD have longer runnng tme per teraton than IWFs due to lne searches and the nner optmzaton, respectvely. Both IWFs n [8], however, do not scale well as the number of users, denoted by K, ncreases. The reason s that n each teraton of IWFs, the most recently updated soluton only accounts for a fracton of 1/K n the effectve channels computaton. The authors of [8] proposed a hybrd algorthm as a remedy. But the hybrd algorthm ntroduces addtonal mplementaton complexty and ts performance depends on the emprcal swtch tmng, whch, n turn, are problem specfc. In addton, one of the IWFs n [8], although converges relatvely faster than the other one, requres a storage sze for K nput covarance matrces. These lmtatons of the exstng algorthms motvate us to desgn an effcent and scalable algorthm wth a modest storage requrement for solvng large MIMO BC systems. Our man contrbuton n ths paper s the desgn of a fast algorthm based on the Conjugate Gradent Projecton (CGP) approach. Our algorthm s nspred by [9], where a gradent projecton method was used to solve another nonconvex maxmum sum rate problem for sngle-hop MIMO-based ad hoc networks wth mutual nterference. However, unlke [9], we use conjugate gradent drectons nstead of gradent drectons to elmnate the zgzaggng phenomenon. Also, we develop a rgorous algorthm to solve the projecton problem. Ths s n contrast to [9], where the way of handlng gradent projecton s based on heurstc. Our proposed CGP has the followng attractve features: 1) CGP s extremely fast, and enjoys provable convergence as well as nce scalablty. As opposed to IWFs, the number of teratons requred for convergence of CGP s nsenstve to the ncrease of the number of users. ) CGP has lnear complexty. By adoptng the nexact lne search method called Armjo s Rule, we show that CGP has a comparable complexty to IWFs per teraton, and requres much fewer teratons for convergence n large MIMO BC systems. 3) CGP has a modest memory requrement: It only needs the soluton nformaton from the prevous step, as opposed to IWF, whch requres the soluton nformaton from prevous K 1 steps. 4) CGP s very ntutve and easy to mplement. The remander of ths paper s organzed as follows. In Secton II, we dscuss the network model and formulaton. Secton III ntroduces the key components n our CGP framework, ncludng conjugate gradent computaton and how to perform projecton. We analyze and compare the complexty of CGP wth other algorthms n Secton IV. Numercal results are presented n Secton V. Secton VI concludes ths paper. II. SYSTEM MODEL AND PROBLEM FORMULATION We begn wth ntroducng notaton. We use boldface to denote matrces and vectors. For a complex-valued matrx A, A and A denotes the conjugate and conjugate transpose of

2 A, respectvely. Tr{A} denotes the trace of A. We let I denote the dentty matrx wth dmenson determned from context. A 0 represents that A s Hermtan and postve semdefnte (PSD). Dag{A 1... A n } represents the block dagonal matrx wth matrces A 1,..., A n on ts man dagonal. Suppose that a MIMO Gaussan broadcast channel has K users, each of whch s equpped wth n r antennas, and the transmtter has n t antennas. The channel matrx for user s denoted as H C n r n t. In [] [4], [10], t has been shown that the maxmum sum rate capacty of MIMO BC s equal to the drty-paper codng regon, whch can be computed by solvng the optmzaton problem as follows: Maxmze =1 log det(i+h( Γj)H ) det(i+h ( 1 Γj)H ) subject to Γ 0, = 1,,..., K =1 Tr(Γ ) P, where Γ C n t n t, = 1,..., K, are the downlnk nput covarance matrces, P represents the maxmum transmt power at the transmtter. It s evdent that (1) s a nonconvex optmzaton problem. However, the authors n [], [4] showed that due to the dualty between MIMO BC and MIMO MAC, (1) s equvalent to the followng MIMO MAC problem wth a sum power constrant: ( Maxmze log det I + ) K =1 H Q H subject to Q 0, = 1,,..., K () =1 Tr(Q ) P, where Q C n r n r, = 1,..., K are the uplnk nput covarance matrces. For convenence, we use the matrx Q = [ ] Q 1 Q... Q K to denote the set of all uplnk ( nput covarance matrces, and let F (Q) = log det I + ) K =1 H Q H represent the objectve functon of (). After solvng (), we can recover the solutons of (1) va approprate mappng []. III. SOLUTION PROCEDURE In ths paper, we propose an effcent algorthm based on conjugate gradent projecton (CGP) to solve (). CGP utlzes the powerful concept of Hessan conjugate to deflect the gradent drecton approprately so as to acheve the superlnear convergence rate [11]. Ths s somewhat smlar to the wellknown quas-newton methods (e.g., BFGS method). CGP follows the same dea of gradent projecton whch was orgnally proposed by Rosen [1]. Durng each teraton, CGP projects the conjugate gradent drecton to fnd an mprovng feasble drecton. Its convergence proof can be found n [11]. The framework of CGP for solvng () s shown n Algorthm 1. Due to the complexty of the objectve functon n (), we adopt an nexact lne search method called Armjo s Rule to avod excessve objectve functon evaluatons, whle stll enjoyng provable convergence [11]. The basc dea of Armjo s Rule s that at each step of the lne search, we sacrfce accuracy for effcency as long as we have suffcent (1) Algorthm 1 Conjugate Gradent Projecton Method Intalzaton: Choose the ntal condtons Q (0) = [Q (0) 1, Q(0),..., Q(0) K ]T. Let k = 0. Man Loop: 1. Calculate the conjugate gradents G (k), = 1,,..., K.. Choose an approprate step sze s k. Let Q (k) = Q (k) + s k G (k), for = 1,,..., K. 3. Let Q (k) be the projecton of Q (k) onto Ω + (P ), where Ω + (P ) {Q, = 1,..., K Q 0, =1 Tr{Q } P }. 4. Choose approprate step sze α k. Let Q (k+1) l Q (k) ), = 1,,..., K. = Q (k) l + α k ( (k) Q 5. k = k + 1. If the maxmum absolute value of the elements n Q (k) Q (k 1) < ɛ, for = 1,,..., L, then stop; else go to step 1. mprovement. Accordng to Armjo s Rule, n the k th teraton, we choose σ k = 1 and α k = β m k (the same as n [9]), where m k s the frst non-negatve nteger m that satsfes F (Q (k+1) ) F (Q (k) ) σβ m G (k), Q (k) Q (k) K [ ( )] = σβ m Tr Q(k) Q (k), (3) =1 G (k) where 0 < β < 1 and 0 < σ < 1 are fxed scalars. Next, we wll consder two major components n the CGP framework: 1) how to compute the conjugate gradent drecton G, and ) how to project Q (k) onto the set Ω + (P ) {Q, = 1,..., K Q 0, =1 Tr{Q } P }. A. Computng the Conjugate Gradents The gradent Ḡ Q F (Q) depends on the partal dervatves of F (Q) wth respect to Q. By usng the formula ln det(a+bxc) X = [ C(A + BXC) 1 B ] T [13] and lettng A = I+,j H j Q jh j, B = H, X = Q, and C = H, we can compute the partal dervatve of F (Q) wth respect to Q as follows: F (Q) = log det I + H j Q Q Q jh j = H I + H j Q jh j 1 H j T. (4) Further, snce z f(z) = ( f(z)/ z) [14], we have 1 ( ) F (Q) Ḡ = = H I + H j Q Q jh j H. (5) Then, the conjugate gradent drecton can be computed as G (k) = Ḡ(k) +ρ k G (k 1). In ths paper, we adopt the Fletcher and Reeves choce of deflecton [11], whch can be computed as ρ k = Ḡ(k) Ḡ(k 1). (6)

3 The purpose of deflectng the gradent usng (6) s to fnd G (k), whch s the Hessan-conjugate of G (k 1). By dong ths, we can elmnate the zgzaggng phenomenon encountered n the conventonal gradent projecton method, and acheve the superlnear convergence rate [11] wthout actually storng a large Hessan approxmaton matrx as n quas-newton methods. B. Performng Projecton In ths secton, we develop a rgorous algorthm for the problem of projectng the conjugate gradents onto the constrant set n the dual MIMO MAC problem. Ths s n contrast to the heurstc method n [9], where the authors smply set the frst dervatve to zero to get the soluton when solvng the constraned Lagrangan dual of the projecton problem. Frst, notng from (5) that G s Hermtan, we have that s Hermtan as well. Then, the projecton problem becomes how to smultaneously project a set of K Hermtan matrces onto the set Ω + (P ), whch contans a constrant on sum power for all users. Ths s dfferent to [9], where the projecton was performed on each ndvdual power constrant. Our approach s to construct a block dagonal matrx D = Dag{Q 1... Q K } C (K n r) (K n r ). It s easy Q (k) = Q (k) + s k G (k) to recognze that f Q Ω + (P ), = 1,..., K, we have Tr(D) = =1 Tr (Q ) P, and D 0. In ths paper, we use Frobenus norm, denoted by F, as the matrx dstance crteron. The dstance between two matrces A and B s defned as A B F = ( Tr [ (A B) (A B) ]) 1. Thus, gven a block dagonal matrx D, we wsh to fnd a matrx D Ω + (P ) such that D mnmzes D D F. For more convenent algebrac manpulatons, we nstead study the followng equvalent optmzaton problem: Mnmze 1 D D F subject to Tr( D) P, D 0. (7) In (7), the objectve functon s convex n D, the constrant D 0 represents the convex cone of postve semdefnte matrces, and the constrant Tr( D) P s a lnear constrant. Thus, the problem s a convex mnmzaton problem and we can exactly solve ths problem by solvng ts Lagrangan dual problem. Assocatng Hermtan matrx X to the constrant D 0, µ to the constrant Tr( D) P, we can wrte the Lagrangan as g(x, µ) = mn D { 1 D D F Tr(X D) ( + µ Tr( D) P )}. (8) Snce g(x, µ) s an unconstraned convex quadratc mnmzaton problem, we can compute the mnmzer of (8) by smply settng the dervatve of (8) (wth respect to D) to zero,.e., ( D D) X + µi = 0. Notng that X = X, we have D = D µi + X. Substtutng D back nto (8), we have g(x, µ) = 1 X µi F µp + Tr [(µi X) (D + X µi)] = 1 D µi + X F µp + 1 D F. (9) Therefore, the Lagrangan dual problem can be wrtten as Maxmze 1 D µi + X F µp + 1 D F subject to X 0, µ 0. (10) After solvng (10), we can have the optmal soluton to (7) as D = D µ I + X, (11) where µ and X are the optmal dual solutons to Lagrangan dual problem n (10). Although the Lagrangan dual problem n (10) has a smlar structure as that n the prmal problem n (7) (havng a postve semdefntve matrx constrant), we fnd that the postve semdefnte matrx constrant can ndeed be easly handled. To see ths, we frst ntroduce Moreau Decomposton Theorem [15] from convex analyss. Theorem 1: (Moreau Decomposton) Let K be a closed convex cone. For x, x 1, x C p, the two propertes below are equvalent: 1) x = x 1 + x wth x 1 K, x K o and x 1, x = 0, ) x 1 = p K (x) and x = p K o(x), where K o {s C p : s, y 0, y K} s called the polar cone of cone K, p K ( ) represents the projecton onto cone K. In fact, the projecton onto a cone K s analogous to the projecton onto a subspace. The only dfference s that orthogonal subspaces s replaced by polar cones. Now we consder how to project a Hermtan matrx A C n n onto the postve and negatve semdefnte cones. Frst, we can perform egenvalue decomposton on A yeldng A = U A Dag{λ, = 1,..., n}u A, where U A s the untary matrx formed by the egenvectors correspondng to the egenvalues λ, = 1,..., n. Then, we have the postve semdefnte and the negatve semdefnte projectons of A as follows: A + = U A Dag{max{λ, 0}, = 1,,..., n}u A, (1) A = U A Dag{mn{λ, 0}, = 1,,..., n}u A. (13) The proof of (1) and (13) s a straghtforward applcaton of Theorem 1 by notng that A + 0, A 0, A +, A = 0, A + + A = A, and the postve semdefnte cone and negatve semdefnte cone are polar cones to each other. We now consder the term D µi + X, whch s the only term nvolvng X n the dual objectve functon. We can rewrte t as D µi ( X), where we note that X 0. Fndng a negatve semdefnte matrx X such that D µi ( X) F s mnmzed s equvalent to fndng the projecton of D µi onto the negatve semdefnte cone. From the prevous dscussons, we mmedately have X = (D µi). (14) Snce D µi = (D µi) + + (D µi), substtutng (14) back to the Lagrangan dual objectve functon, we have mn X D µi + X F = (D µi) +. (15)

4 Thus, the matrx varable X n the Lagrangan dual problem can be removed and the Lagrangan dual problem can be rewrtten as (D µi) µp F D F Maxmze ψ(µ) 1 subject to µ 0. (16) Suppose that after performng egenvalue decomposton on D, we have D = UΛU, where Λ s the dagonal matrx formed by the egenvalues of D, U s the untary matrx formed by the correspondng egenvectors. Snce U s untary, we have (D µi) + = U (Λ µi) + U. It then follows that (D µi) + F = (Λ µi) + F. (17) We denote the egenvalues n Λ by λ, = 1,,..., K n r. Suppose that we sort them n non-ncreasng order such that Λ = Dag{λ 1 λ... λ K nr }, where λ 1... λ K nr. It then follows that (Λ µi)+ F = K nr (max {0, λ j µ}). (18) From (18), we can rewrte ψ(µ) as ψ(µ) = 1 K n r (max {0, λ j µ}) µp + 1 D n F. (19) It s evdent from (19) that ψ(µ) s contnuous and (pece-wse) concave n µ. Generally, pece-wse concave maxmzaton problems can be solved by usng the subgradent method. However, due to the heurstc nature of ts step sze selecton strategy, subgradent algorthm usually does not perform well. In fact, by explotng ts specal structure, (16) can be solved effcently. We can search the optmal value of µ as follows: Let ndex the peces of ψ(µ), = 0, 1,..., K n r. Intally we set = 0 and ncrease subsequently. Also, we ntroduce λ 0 = and λ K nr+1 =. We let the endpont objectve value ψ (λ 0 ) = 0, φ = ψ (λ 0 ), and µ = λ 0. If > K n r, the search stops. For a partcular ndex, by settng µ ψ (ν) 1 (λ µ) µp = 0, (0) µ we have =1 µ = =1 λ P. (1) Now we consder the followng two cases: 1) If µ [ λ+1, λ] R+, where R + denotes the set of non-negatve real numbers, then we have found the optmal soluton for µ because ψ(µ) s concave n µ. Thus, the pont wth zero-value frst dervatve, f t exsts, must be the unque global maxmum soluton. Hence, we can let µ = µ and the search s done. ) If µ / [ λ+1, λ] R+, we must have that the local maxmum n the nterval [ λ+1, λ] R+ s acheved at one of the two endponts. Note that the objectve value ( ) ψ λ has been computed n the prevous teraton. Ths s because from ( the ) contnuty ( of the ) objectve functon, we have ψ λ = ψ 1 λ. Thus, we only ( need) to compute ( the) other endpont ( ) objectve value ψ λ+1. If ψ λ+1 < ψ λ = φ, then we know µ s the optmal soluton; else let µ = λ+1, ( ) φ = ψ λ+1, = + 1 and contnue. Snce there are K n r + 1 ntervals n total, the search process takes at most K n r + 1 steps to fnd the optmal soluton µ. Hence, ths search s of polynomal-tme complexty O(n r K). After fndng µ, we can compute D as D = (D µ I) + = U (Λ µ I) + U. () That s, the projecton D can be computed by adjustng the egenvalues of D usng µ and keepng the egenvectors unchanged. The projecton of D n onto Ω + (P ) s summarzed n Algorthm. Algorthm Performng Projecton Intaton: 1. Construct a block dagonal matrx D. Perform egenvalue decomposton D = UΛU, sort the egenvalues n non-ncreasng order.. Introduce λ 0 = and λ K nt +1 =. Let = 0. Let the endpont objectve value ψ (λ 0 ) = 0, φ = ψ (λ 0 ), and µ = λ 0. Man Loop: 1. If > K n r, go to the fnal step; else let µ = ( λ j P )/.. If µ [λ +1, λ ] R +, then let µ = µ and go to the fnal step. 3. Compute ψ(λ+1 ). If ψ(λ+1 ) < φ, then go to the fnal step; else let µ = λ+1, φ = ψ(λ+1 ), = + 1 and contnue. Fnal Step: Compute D as D = U (Λ µ I) + U. IV. COMPLEXITY ANALYSIS In ths secton, we compare our proposed CGP wth exstng methods for solvng MIMO BC. Smlar to IWFs [8], SD [6], and DD [7], CGP has lnear complexty property. Although CGP also needs to compute gradents n each teraton, the computaton s much easer than that n SD due to the dfferent perspectves n handlng MIMO BC. In ths paper, we compare CGP wth IWF (Algorthms 1 and n [8]) as IWF has the least complexty among aforementoned algorthms. For convenence, we wll refer to Algorthm 1 and Algorthm n [8] as IWF1 and IWF, respectvely. TABLE I PER ITERATION COMPLEXITY COMPARISON BETWEEN CGP AND IWFS CGP IWFs Gradent/Effectve Channel K K Lne Search O(mK) N/A Projecton/Water-Fllng O(n rk) O(n rk) Overall O((m n r )K) O(( + n r )K) To better llustrate the comparson, we lst the complexty per teraton for each component of CGP and IWFs n Table I. For both CGP and IWFs, t can be seen that the most tme-consumng part (ncreasng wth respect to K) s the addtons of the terms n the form of H Q H when computng gradents and effectve channels. Snce the term

5 (I + =1 H Q H ) s common to all gradents, we only need to compute ths sum once n each teraton. Thus, the number of such addtons per teraton for CGP s K. In IWF1 and IWF, the number of such addtons can be reduced to K by a clever way of mantanng a runnng sum of (I + j H j Q jh j ). However, the runnng sum, whch requres K addtons for IWF1, stll needs to be computed n the ntalzaton step. On the other hand, although the basc deas of the projecton n CGP and water-fllng are dfferent, the algorthm structures of them are very smlar and they have exactly the same complexty of O(n r K). The only unque component n CGP s the lne search step, whch has the complexty of O(mK) (n terms of the addtons of H Q H terms), where m s the number of trals n Armjo s Rule. Therefore, the overall complexty per teraton for CGP and IWFs are O((m n r )K) and O(( + n r )K), respectvely. Accordng to our computatonal experence, the value of m usually les wthn two and four. Thus, when n r s large (e.g., n r 4), the overall complexty per teraton for CGP and IWFs are comparable. In the next secton, however, we wll show that the numbers of teratons requred for convergence n CGP s much less than that n IWFs for large MIMO BC systems, and t s nsenstve to the ncrease of the number of users. Moreover, CGP has a modest memory requrement: It only requres the soluton nformaton from the prevous step, as opposed to IWF1, whch requres prevous K 1 steps. Ths means that IWF1 requres a storage sze for K nput covarance matrces. V. NUMERICAL RESULTS We show results for a large MIMO BC system consstng of 100 users wth n t = n r = 4. The convergence processes are plotted n Fg. 1. It s observed from Fg. 1 that CGP takes only 9 teratons to converge and t outperforms both IWFs. IWF1 s convergence speed sgnfcantly drops after ts ntal mprovement. It s also seen that IWF s performance s nferor to IWF1, and ths observaton s consstent to the results n [8]. Both IWF1 and IWF fal to converge wthn 100 teratons. The scalablty problem for both IWFs s not surprsng because for both algorthms, the most recently updated covarance matrces only account for a fracton of 1/K n the effectve channels computaton, whch means t does not effectvely make use of the most recent soluton. In all of our numercal examples wth dfferent number of users, CGP s found to converge wthn 30 teratons. VI. CONCLUSION In ths paper, we developed an effcent algorthm based on conjugate gradent projecton (CGP) for solvng the maxmum sum rate problem of MIMO BC. We analyzed ts complexty and convergence behavor and showed that CGP enjoys provable convergence, scalablty, and effcency. The attractve features of CGP and encouragng results ndcate that CGP s an excellent method for solvng the maxmum sum rate problem for large MIMO BC systems. Sum Rate (nats/s/hz) Number of Iteratons Fg. 1. Comparson n a 100-user MIMO BC channel wth n t = n r = 4. ACKNOWLEDGEMENTS The work of Y.T. Hou and J. Lu has been supported n part by the Offce of Naval Research (ONR) under Grant N The work of H.D. Sheral has been supported n part by NSF Grant REFERENCES [1] H. Wengarten, Y. Stenberg, and S. Shama (Shtz), The capacty regon of the Gaussan multple-nput multple-output broadcast channel, IEEE Trans. Inf. Theory, vol. 5, no. 9, pp , Sep [] S. Vshwanath, N. Jndal, and A. Goldsmth, Dualty, achevable rates, and sum-rate capacty of MIMO broadcast channels, IEEE Trans. Inf. Theory, vol. 49, no. 10, pp , Oct [3] P. Vswanath and D. N. C. Tse, Sum capacty of the vector Gaussan broadcast channel and uplnk-downlnk dualty, IEEE Trans. Inf. Theory, vol. 49, no. 8, pp , Aug [4] W. Yu, Uplnk-downlnk dualty va mnmax dualty, IEEE Trans. Inf. Theory, vol. 5, no., pp , Feb [5] T. Lan and W. Yu, Input optmzaton for mult-antenna broadcast channels and per-antenna power constrants, n Proc. IEEE GLOBECOM, Dallas, TX, U.S.A., Nov. 004, pp [6] H. Vswanathan, S. Venkatesan, and H. Huang, Downlnk capacty evaluaton of cellular networks wth known-nterference cancellaton, IEEE J. Sel. Areas Commun., vol. 1, no. 5, pp , Jun [7] W. Yu, A dual decomposton approach to the sum power Gaussan vector multple-access channel sum capacty problem, n Proc. Conf. Informaton Scences and Systems (CISS), Baltmore, MD, U.S.A., 003. [8] N. Jndal, W. Rhee, S. Vshwanath, S. A. Jafar, and A. Goldsmth, Sum power teratve water-fllng for mult-antenna Gaussan broadcast channels, IEEE Trans. Inf. Theory, vol. 51, no. 4, pp , Apr [9] S. Ye and R. S. Blum, Optmzed sgnalng for MIMO nterference systems wth feedback, IEEE Trans. Sgnal Process., vol. 51, no. 11, pp , Nov [10] G. Care and S. Shama (Shtz), On the achevable throughput of a multantenna Gaussan broadcast channel, IEEE Trans. Inf. Theory, vol. 49, no. 7, pp , Jul [11] M. S. Bazaraa, H. D. Sheral, and C. M. Shetty, Nonlnear Programmng: Theory and Algorthms, 3rd ed. New York, NY: John Wley & Sons Inc., 006. [1] J. B. Rosen, The gradent projecton method for nonlnear programmng, Part I, Lnear constrants, SIAM J. Appled Mathematcs, vol. 8, pp , [13] J. R. Magnus and H. Neudecker, Matrx Dfferental Calculus wth Applcatons n Statstcs and Economcs. New York: Wley, [14] S. Haykn, Adaptve Flter Theory. Englewood Clffs, NJ: Prentce-Hall, [15] J.-B. Hrart-Urruty and C. Lemaréchal, Fundamentals of Convex Analyss. Berln: Sprnger-Verlag, 001.