Renormalised steepest descent converges to a two-point attractor

Transcription

1 Renormalsed steepest descent converges to a two-pont attractor Luc Pronzato, Henry P. Wynn and Anatoly A. Zhglavsky Abstract The behavour of the standard steepest-descent algorthm for a quadratc functon n IR d s nvestgated. We show that by rescalng the terates to reman always on the unt sphere one can reveal specal features of ths behavour. The renormalzed algorthm s shown to converge to a two-pont cycle on the unt crcle. The cycle depends on the startng pont n a complcated manner (the set of ponts convergng to the same cycle s fractal), but all cycles belong to a partcular plane, gven by certan egenvectors of the Hessan matrx of the obectve functon. The stablty of the attractor s analysed. The rate of convergence of the algorthm s nvestgated. It s shown that the worst value of ths rate s obtaned only for some partcular startng ponts. The ntroducton of a relaxaton coeffcent n the steepest-descent algorthm completely changes ts behavour, whch may become chaotc. Dfferent attractors are presented. We show that relaxaton allows a sgnfcantly mproved rate of convergence. 1 Introducton: an unsolved problem For a general smooth functon f( ) the steepest-descent algorthm s where s the k-th terate of the algorthm, x (k+1) = x (k) α k f(x (k) ), (1) x (k) = ( x (k) 1,..., x (k) ) T d f(x) = ( f,..., f ) T x 1 x d Ths work was supported by a UK Engneerng and Physcal Scences Research Councl for the second author and a grant of the French Mnstère de l Éducaton Natonale (nvtaton trennale, procédure PAST) for the thrd author. Dr., Laboratore I3S, CNRS UPRES-A 6070, Sopha Antpols, Valbonne, France Prof., Dept. of Statstcs, Unversty of Warwck, Coventry CV4 7AL, UK Prof., Dept. of Mathematcs, St. Petersburg Unversty, Bblotechnaya sq. 2, , Russa 1

2 s the gradent of f( ), and α k = arg mn α f(x (k) α f(x (k) ). In [?] the authors studed the asymptotc behavour of ths algorthm for a quadratc functon n IR d : f(x) = 1 2 (x x ) T A(x x ), where x, x IR d and A s a postve defnte d d matrx. Ths seemngly smple problem has some very complex aspects whch are uncovered by the followng renormalzaton dea. Defne v (k) = x(k) x x (k) x, (2) whch has the effect of rescalng the terates to reman always on the unt sphere: v = 1. The authors studed the behavour of the process {v (k) } as k. Except for pathologcal cases, n the lmt the process s found to belong to the two dmensonal space spanned by the egenvectors (u 1, u d ) correspondng to the mnmal and maxmum egenvalues λ 1 and λ d of the matrx A under the assumpton 0 < λ 1 < λ 2... λ d 1 < λ d, (3) where λ s the -th ordered egenvalue of A. Orderng the v (k) compatbly wth (3), the conecture states that v (k) 0 for = 2,..., d 1 as k, and that the algorthm, n ts renormalsed form, converges to a two-pont set on the crcle { v v d 2 = 1} n the plane spanned by u 1 and u d. Numercal smulatons show that the same convergence property holds for a convex dfferentable functon locally quadratc around ts mnmum x. It has been known for many years that the asymptotc convergence rate of the algorthm depends on the rato ρ = λ d /λ 1 (for nstance, see [?], p. 152). Thus for any x (k) IR d ( ) 2 ρ 1 f(x (k+1) ) f(x (k) ) (4) ρ + 1 (see also Secton 4). Despte an ntensve search, no more precse propertes concernng the rate of convergence were found n the lterature. The authors doubts about the exstence of results on the asymptotc behavour of the steepest-descent algorthm were ncreased by the complexty of the behavour of the renormalzed algorthm en route to the lmtng crcle. Ths behavour s fractal n nature, and t stll remans open how the lmtng asymptotc rate depends on the startng values. It s the worst-case rate, gven by (4), not the actual rate, whch smply depends on λ 1 and λ d. However we are able to show that the algorthm attracts to two conugate ponts on the crcle. It s easy to check that startng on the crcle the renormalzed algorthm oscllates between the ponts, but wthout attracton to the crcle ths lmtng behavour s only a conecture. We shall return to the dscusson of these conugate ponts and the correspondng asymptotc rate of convergence n Sectons 3 and 4 respectvely. The behavour of the steepest-descent algorthm wth relaxaton s consdered n Secton 5. 2

3 2 Attracton theorem Theorem 1 Let the obectve functon f be f(x) = 1 2 (x x ) T A(x x ), (5) where A s a postve defnte matrx wth ordered egenvalues 0 < λ 1 < λ 2... λ d 1 < λ d. Let V = span(u 1, u d ) be the two-dmensonal plane generated by the (dstnct orthogonal) egenvectors u 1, u d correspondng to λ 1 and λ d, respectvely. Then for any startng vector x (1) for whch u T 1 v (1) 0 and u T d v (1) 0 (6) the algorthm attracts to the plane V n the followng sense: w T v (k) 0 k for any non-zero vector w V, where v (k) s defned by (2). Moreover, the sequence {v (k) } converges to a two-pont cycle. Proof. For the quadratc functon (5) the gradent of f( ) at x (k) equals and g (k) = f(x (k) ) = A(x (k) x ), α k = (g(k) ) T g (k) (g (k) ) T Ag (k). Therefore, the algorthm (1) can be rewrtten as x (k+1) = x (k) (g(k) ) T g (k) (g (k) ) T Ag (k) g(k). (7) Wthout loss of generalty we can assume that x = 0 and the matrx A s dagonal: A = dag(λ 1,... λ d ) where 0 < λ 1 < λ 2... λ d 1 < λ d. Wth ths assumpton, the teraton of the algorthm (7) can be wrtten as x (k+1) = x (k) d=1 (g (k) ) 2 d=1 λ (g (k) ) 2 g(k) for = 1,..., d. (8) Multplyng both sdes of -th equaton of (8) by λ we obtan g (k+1) = g (k) d=1 (g (k) ) 2 d=1 λ (g (k) ) λ g (k) 2 for = 1,..., d. (9) 3

4 Introduce now the varables and ther renormalsed versons Note that and y (k) = = (g (k) ) 2, 0 for = 1,..., d and d =1 z (k+1) y (k+1) = = 1 λ d=1 y (k) d=1 λ y (k) y (k) d=1. (10) y (k) ( d=1 λ λ ) 2 dl=1 ( d=1 λ 2 = 1. The equatons (9) then mply y (k) for = 1,..., d, λ l ) 2 z (k) l for = 1,..., d. (11) Note that the updatng formula (11) s exactly the same for the weghts and correspondng to equal λ = λ, and therefore these weghts can be summed. It follows that a proof based on the assumpton that all λ are dfferent can easly be extended to the case of tes among λ 2,..., λ d 1. We thus assume that all egenvalues of A are dfferent: 0 < λ 1 < λ 2 <... < λ d. Note also that to prove the convergence to the two-dmensonal plane V for the sequence {x (k) / x (k) }, t s enough to prove the same property for the sequence { }, k. We dvde the rest of the proof nto four parts: we show () that the sequence (11) converges to a two-dmensonal plane, () that the drectons are eventually fxed, and () that they fx at u 1 and u d. Fnally, we show (v) the convergence to a two-pont cycle. It s convenent to consder the dscrete measures π k = { λ1... λ d 1... d whch place the weghts at the ponts λ, respectvely ( = 1,..., d). These measures carry all nformaton about the sequence (11). Let ψ( ) denote the operator correspondng to the applcaton of (11) to a measure π, so that π k+1 = ψ(π k ). (). Defne the moments µ m = µ m (π k ) = λ m dπ k (λ) = d =1 } λ m, m = 0, ±1, ±2,... (12) so that µ 0 = 1, µ 1 = d =1 λ, etc. Defne also the moment matrces M n = M n (π k ) by {M n },l = µ +l 2 (, l = 1,..., n) so that M 2 (π k ) = ( µ0 µ 1 µ 1 µ 2 ) and M 3 (π k ) = 4 µ 0 µ 1 µ 2 µ 1 µ 2 µ 3 µ 2 µ 3 µ 4.

5 Then the denomnator n the rght hand sde of (11) s D k = d d l=1 =1 λ 2 λ l Usng the updatng formula (11) we have Therefore D k+1 = d d l=1 =1 λ z (k+1) 2 λ l l = µ 2 µ 2 1 = det[m 2 (π k )]. z (k+1) l = d =1 λ 2 z (k+1) d =1 = 1 D k (µ 2 1µ 2 2µ 1 µ 3 + µ 4 ) 1 D 2 k (µ 3 1 2µ 1 µ 2 + µ 3 ) 2. λ z (k+1) D k+1 D k = 2µ 1µ 2 µ 3 + µ 2 µ 4 µ 2 1µ 4 µ 2 3 µ 2 2 (µ 2 µ 2 1) 2 = det[m 3(π k )] {det[m 2 (π k )]} 2. The condtons (6) transfer through the updatng formula (11) to 1 > 0 and d > 0 for every k = 1, 2,... From smple moment theory we have therefore that the matrx M 2 (π k ) s postve defnte for every k = 1, 2,... whch mples D k = det[m 2 (π k )] > 0. Also, M 3 (π k ) s a non-negatve defnte moment matrx and det[m 3 (π k )] 0. Thus D k+1 D k 0 whch means that the sequence {D k } s monotonously non-decreasng. Snce π k s a probablty measure wth bounded support, D k = det[m 2 (π k )] s bounded above by some constant D and therefore the sequence {D k } converges monotoncally to a lmt and D k+1 D k 0 when k. Moreover, det[m 3 (π k )] = (D k+1 D k )D 2 k (D k+1 D k )D 2, so that det[m 3 (π k )] 0 when k. Note that usng the Bnet-Cauchy lemma D k = det[m 2 (π k )] = < det[m 3 (π k )] = <<l (λ λ ) 2 D 1 > 0, 2 l (λ λ ) 2 (λ l λ ) 2 (λ λ l ) 2 0, k. (13) For a fxed teraton k defne the par ( k, k ) whch acheves max <. (If there are several such pars, take the smallest of them n, say, lexcographcal order.) Then for every k we have Therefore δ k D 1 D k k k, δ < k k < (λ λ ) 2. < 1 δ, δ < k < 1 δ, (14) 5

6 where δ = D 1 <(λ λ ) 2 > 0. (15) From (13) we have det[m 3 (π k )] k k (λ k λ k ) 2 (λ λ k ) 2 (λ λ k ) 2. k, k Snce all λ are dstnct, det[m 3 (π k )] 0 and k k δ > 0, we have 0, k. (16) k, k Ths fnshes part () of the proof. The nterpretaton s that although ( k, k ) depends on k the total weght assocated wth all other ponts tends to zero. (). We wll show n addton to (16), that there exsts an teraton number k such that for all k k the par ( k, k ) becomes fxed, that s, ( k, k ) = (, ) for some 1 < d and all k k. Let δd 1 ɛ = (λ d λ 1 ), 2 where δ s defned n (15). Accordng to (16), there exsts k = k (ɛ ) 1 such that the total weght assocated wth all other ponts dfferent from ( k, k ) s smaller than ɛ for all k k. Therefore, for / { k, k } and k k the updatng formula (11) gves From (14), z (k+1) k+1 z (k+1) = ( d=1 λ λ ) 2 < ɛ (λ d λ 1 ) 2 = δ. D k D 1 > δ and z (k+1) k+1 > δ, and therefore / { k, k } mples / { k+1, k+1 }. Ths proves that ( k, k ) = (, ) for some 1 < d and all k k. (). Let us prove that (, ) = (1, d). Recall agan that the assumpton (6) and the updatng formula (11) mply that 1 > 0 and d > 0 for all k. Assume that < d. Denote ɛ = mn{ɛ, δ mn (λ λ )/λ d }, 1 <<d and assume that k = k (ɛ ) k s such that ( k, k ) = (, ) and the total weght assocated wth all other ponts dfferent from (, ) s smaller than ɛ for all k k. The exstence of such a k follows from () and (). For convenence, rewrte the algorthm (11) n the form z (k+1) = (µ 1 λ ) 2 for = 1,..., d, (17) D k 6

7 and note that µ 1 = d =1 λ λ = λ + λ + λ, + λ + λ d λ + λ + ɛ λ d, δλ + (1 δ)λ + ɛ λ d = λ + δ(λ λ ) + ɛ λ d λ δ mn (λ λ ) + ɛ λ d λ. 1 <<d Therefore (17) mples that for all k k z (k+1) d d = (λ d µ 1 ) 2 D k > (λ µ 1 ) 2 D k = z(k+1). We have arrved at a contradcton snce the sequence { } s bounded from below by δ > 0 whle the sequence { d } tends to zero. Thus, = d and analogously = 1. (v). Fnally, let D denote the lmt lm k D k dscussed n (). There are only two dscrete measures π 1, π 2 wth nonzero weghts on λ 1 and λ d and such that namely wth det[m 2 (π 1 )] = det[m 2 (π 2 )] = D, { } π 1 λ1 λ = d, π 2 = p 1 p p = { λ1 λ d 1 p p D (λ d λ 1 ) 2. }, (18) Note that ψ(π 1 ) = π 2 and ψ(π 2 ) = π 1. Therefore, convergence of D k to D mples convergence of π k to the lmtng cycle π 1 π 2 π 1 Remark 1 Theorem 1 obvously generalzes to the case where (6) may not be satsfed. The algorthm then attracts to a two-dmensonal plane V and {v (k) } converges to a twopont cycle. V s defned by the egenvectors u, u assocated wth the smallest and largest egenvalues such that u T v (1) 0, u T v (1) 0. For the sake of smplcty of notatons, we shall assume n what follows that (6) s satsfed, and therefore u = u 1, u = u d. Although the lmtng behavour of the algorthm s smple, ts behavour en route to the attractor s farly complcated and presents a fractal structure. Fgure 1 shows the proecton onto the plane (v 1, v 3 ) of the regon of attracton for v (k) to a small neghbourhood (radus < 0.02) of the pont v = (0.9606, 0, ), wth d = 3, λ 1 = 1, λ 2 = 2, 7

8 λ 3 = 4. It llustrates the dffculty of predctng the lmtng behavour, and thus the aymptotc rate of convergence, see Secton 4, as a functon of the startng pont. Possble locaton of Fgure 1 In Fgure 2, the grey level of startng ponts on the unt sphere depends on the lmtng value of p n (18). Agan, ths llustrates the complexty of the behavour of the algorthm on the way to the attractor. Possble locaton of Fgure 2 3 Stablty of attractors From Theorem 1, the algorthm attracts to two conugate ponts on the crcle { v v d 2 = 1}, characterzed by the dscrete measure (18). However, some values of p correspond to unstable ponts. We shall use the followng defnton of stablty, see [?] p Defnton 1 A fxed pont π for a mappng T ( ) s called stable f ɛ > 0, α > 0 such that π 0 for whch π 0 π < α, T n (π 0 ) π < ɛ for all n > 0. A fxed pont π s unstable f t s not stable. Consder a two-step teraton for, 1 d: = z (k+1) ( d =1 λ z (k+1) λ ) 2 z (k+2) D k+1 = (µ 1 λ ) 2 1 D k D k+1 ( µ 3 1 2µ 1 µ 2 + µ 3 D k λ ) 2, (19) and the correspondng transformaton ψ 2 ( ) defned by π k+2 = ψ 2 (π k ). Snce d =1 = 1 for all k, we substtute 1 d 1 =1 for d n (19). Ths defnes an operator φ( ) : S d 1 S d 1 on the (d 1)-dmensonal canoncal smplex d 1 S d 1 = {z = (z 1,..., z d 1 ) z 0, z 1}, whch maps ( 1,..., d 1 ) to (z(k+2) 1,..., z (k+2) d 1 ). Studyng the propertes of ψ2 ( ) s equvalent to studyng the propertes of φ( ). =1 8

9 Theorem 2 () Stablty. All ponts n the nterval I S defned by where I S =] 1 2 s(λ ), s(λ )[, s(λ) = (λ d λ) 2 + (λ 1 λ) 2 2(λ d λ 1 ) and s such that λ λ 1+λ d 2 s mnmum over all λ s, = 2,..., d 1, are stable. () Instablty. All ponts n the set I U defned by, are unstable. I U = [0, 1 2 s(λ )[ ]1 2 + s(λ ), 1] Proof. () Stablty. Take p I S and consder the fxed pont z(p) = (p, 0,..., 0) for φ( ), z(p) S d 1. Assume that β 1, = 2,..., d 1 and 1 p β 1, that s z(p) β 1. Then z (k+2) = f(λ, p)[1 + O(β 1 )], β 1 0, (20) where wth f(λ, p) = (µ 1 λ) 2 [(µ 1) 3 2µ 1µ 2 + µ 3 D(p)λ] 2 [D(p)] 4 µ m = λ m 1 p + λ m d (1 p), m = 1, 2, 3, D(p) = p(1 p)(λ d λ 1 ) 2. (21) Then, p I S mples f(λ, p) < 1, = 2,..., d 1. Defne f (p) = max {2,...,d 1} f(λ, p) and let K be any constant such that f (p) < K < 1. Then, from (20): and thus β 0 β < β 0, z (k+2) β < β 0, z (k+2m) K, = 2,..., d 1, K m, = 2,..., d 1, m 1. (22) Now, (13) mples det[m 3 (π k )] Cβ 1, wth C some postve constant. Therefore, D k+1 D k Cβ 1 D 2 k. (23) 9

10 Smlarly, z (k+1) z(1 p) β 2 mples D k+2 D k+1 Cβ 2 D 2 k+1 Cβ 2 D 2 k. (24) Snce z(p) β z (k+1) z(1 p) Hβ, wth H some postve constant, (23,24) mply D k+1 D k C β, D k+2 D k+1 C β when z(p) β, wth C = (C/D 2 k) max{1, H}. Ths gves Choosng β < β 0, one then obtan from (22) Therefore, l > 0, D k+2 D k 2C β. D k+2m+2 D k+2m 2C K m β. D k+2l D k 2C β 1 K = O(β), β 0. Moreover, z(p) β D k D(p) Aβ, wth A a postve constant and D(p) gven by (21). Therefore, l > 0, Ths, together wth (22) mples D k+2l D(p) ( 2C 1 K + A ) l > 0, z (k+2l) 1 p < Lβ, L > 1. (25) Fnally, for any ɛ > 0, defne α = mn{ɛ/l, β 0 /L}, (22) and (25) then mply the stablty of the fxed pont z(p). () Instablty. The Jacoban matrx J φ of φ( ) at ponts z(p) = (p, 0,..., 0) can be computed analytcally, and s gven by: (J φ ) = φ (z) = z z=z(p) 1 f = = 1 (λ λ 1 )(λ d λ ) 2 [2λ d (1 p)+2λ 1 p λ 1 λ ] p(1 p) 2 (λ d λ 1 ) 4 f > 1 and = 1 β. [λ d (1 p)+λ 1 p λ ] 2 [λ d p+λ 1 (1 p) λ ] 2 p 2 (1 p) 2 (λ d λ 1 ) 4 f = 0 otherwse. One can easlly check that for p I U, (J φ ) > 1 for at least one n {2,..., d 1}. Snce the (J φ ) s are egenvalues of J φ, Theorem n [?] ndcates that z(p) s unstable. 10

11 Note that λ 1,..., λ d the stablty nterval I S contans the nterval ] , [ ] , [. 2 When d = 3, numercal smulatons show that for any ntal densty of x (1) n IR d assocated wth a densty of z (1) on S d 1 reasonably spread, the densty of attractors z(p) characterzed by p can be approxmated by the densty ϕ(p) = C log mn{1, (J φ ) 22 } = { C log(jφ ) 22 f p I A 0 otherwse, where C s a normalsaton constant. Fgure 3 shows the emprcal densty of attractors (full lne) together wth ϕ(p) (dashed lne) n the case λ 1 = 1, λ 2 = 4, λ 3 = 10. The support of ths densty concdes wth the stablty nterval I S. Possble locaton of Fgure 3 When d > 3, the densty of attractors depends on the ntal densty of x (1). 4 Rate of convergence Defne the convergence rate at teraton k by r k = f(x(k+1) ) f(x (k) ), (26) where f(x) s gven by (5). Wthout any loss of generalty, one can take f(x) = d =1 λ x 2. Rewrtng r k n terms of defned by (10), one gets r k = 1 1 µ 1 µ 1, where µ m s defned by (12). Defne the asymptotc rate as R = R(x (1), x ) = lm 1/k k r k =1. (27) Generally, R depends on the ntal pont x (1) and the optmal pont x. Theorem 1 mples that for any fxed x and almost all x (1) the asymptotc rate depends only on the attractor (18) and s gven by ( f(x (k+2) ) 1/2 ) R(p) = f(x (k) ) 11

12 where x (k) corresponds to π 1 or π 2, see (18). Ths gves R(p) = p(1 p)(ρ 1) 2 [p + ρ(1 p)][(1 p) + ρp], wth ρ = λ d /λ 1 the condton number of the matrx A. The functon R(p) s symmetrc wth respect to 1/2 and monotonously ncreasng from 0 to 1/2. The worst asymptotc rate s thus obtaned at p = 1/2: R max = ( ) 2 ρ 1, ρ + 1 see (4). Note that from Kantorovch nequalty, see [?], p. 151, µ 1 µ 1 (1 + ρ) 2 /(4ρ), and therefore x (k), r k R max. The worst rate s thus acheved only when x (k) 1 = ±ρx (k) d, x (k) 2 = = x (k) d 1 = 0. Consder now another convergence rate, defned by where R = lm Rewrtng r k n terms of, one gets 1/k k r k =1 r k = x(k+1) 2 x (k) 2. r k = 1 2µ 1 µ 1 µ µ 2 1µ 2. One can easlly check that for almost all x (1) the asymptotc rate R s equal to R(p), where p defnes the attractor. 5 Steepest descent wth relaxaton The ntroducton of a relaxaton coeffcent γ, wth 0 < γ < 1, n the steepest-descent algorthm totally changes ts behavour. The algorthm (7) then becomes x (k+1) = x (k) γ (g(k) ) T g (k) (g (k) ) T Ag (k) g(k). For fxed A, dependng on the value of γ, the renormalzed process ether attracts to perodc orbts (the same for almost all startng ponts) or exhbts a chaotc behavour. Fgures 4 (respectvely 5) presents the classcal perod-doublng phenomenon n the case d = 2 when λ 1 = 1 and λ 2 = 4 (respectvely λ 2 = 10). Fgures 6 (respectvely 7) gve the asymptotc rate (27) as a functon of γ n the same stuaton. 12.

13 Possble locaton of Fgure 4 Possble locaton of Fgure 5 Possble locaton of Fgure 6 Possble locaton of Fgure 7 We get now nstead of (26): r k (γ) = 1 γ(2 γ) µ 1 µ 1. Note that from Kantorovch nequalty the worst value of the rate s 4ρ 1 γ(2 γ) (1 + ρ) > R 2 max, f γ < 1. However, numercal results show that for γ large enough the asymptotc rate s sgnfcantly better than R max. A detaled analyss of the 2-dmensonal case gves the followng results. () If 0 < γ 2 2ργ, the process attracts to the fxed pont p = 1 and R = R(γ) = 1. ρ+1 ρ+1 2 () If < γ 4ρ, the process attracts to the fxed pont p = 2ρ γ(ρ+1), and ρ+1 (ρ+1) 2 2(ρ 1) R(γ) = R max. () If 4ρ < γ 2( 2+1)ρ (ρ+1) 2 (ρ+1) 2, the process attracts to the 2-pont cycle (p 1, p 2 ), wth p 1,2 = 2ρ γ(ρ + 1) ± γ(γρ 2 4ρ + 2γρ + γ), 2(ρ 1) and R(γ) = 1 γ. (v) For larger values of γ one observes a classcal perod-doublng phenomenon, see Fgures 4 and 5. 13

14 (v) If ρ > , the process attracts agan to a 2-pont cycle for values of γ larger than γ ρ = 8ρ, see Fgure 5. For the lmtng case γ = γ (ρ+1) 2 ρ, the cycle s gven by (p 1, p 2), wth p 1,2 = ρ ( ρ 2 2ρ + 5 ± 2 (ρ 2 2ρ + 5)(5ρ 2 2ρ + 1) ), (ρ 1)(ρ + 1) 3 and the assocated asymptotc rate s R(γ ρ ) = ρ2 6ρ + 1 ρ 2 1 In hgher dmensons, repeated numercal trals show that the process typcally no longer attracts to the 2-dmensonal plane spanned by (u 1, u d ). Fgure 8 presents the proecton of the attractor of on the plane (z 1, z 3 ) n the case d = 3, λ 1 = 1, λ 2 = 2, λ 3 = 4 and γ = Such a pcture s typcal for d > 2. Possble locaton of Fgure 8. 14

15 Fgure captons Fgure 1. Proecton of the regon of attracton for v (k) to a small neghbourhood (radus < 0.02) of the pont v = (0.9606, 0, ) onto the plane (v 1, v 3 ) (d = 3, λ 1 = 1, λ 2 = 2, λ 3 = 4). Fgure 2. Startng ponts on the unt sphere colored a a functon of the lmtng value of p n (18) (d = 3, λ 1 = 1, λ 2 = 2, λ 3 = 4). Fgure 3. Emprcal densty of attractors (full lne) and ϕ(p) (dashed lne) (d = 3, λ 1 = 1, λ 2 = 4, λ 3 = 10). Fgure 4. Attractors for z (1) as a functon of γ (d = 2, λ 1 = 1, λ 2 = 4). Fgure 5. Attractors for z (1) as a functon of γ (d = 2, λ 1 = 1, λ 2 = 10). Fgure 6. Asymptotc rate (27) as a functon of γ (d = 2, λ 1 = 1, λ 2 = 4). Fgure 7. Asymptotc rate (27) as a functon of γ (d = 2, λ 1 = 1, λ 2 = 10). Fgure 8. Proecton of the attractor of on the plane (z 1, z 3 ) (d = 3, λ 1 = 1, λ 2 = 2, λ 3 = 4, γ = 0.97). 15