Frames and pseudo-inverses. Ole Christensen 3 October 20, 1994 Abstract We point out some connections between the existing theories for frames and pseudo-inverses. In particular, using the pseudo-inverse of an unbounded operator we show how to make frame-like decompositions with respect to some sequences which do not satisfy the upper frame condition. As a special case this approach immediately gives non trivial results for usual frames. 1 Introduction. The aim of this paper is to exhibit some connections between 2 theories from modern analysis: the frame theory and the theory of pseudo-inverse operators. The frame concept has been introduced in the paper [DS] from 1952, but only in the last 10 years it has been of importance. It is now a very useful tool in wavelet theory ([DGM ], [D]). The theory of pseudo-inverses of operators was founded in the middle of the sixties (see [B]); for matrices about 10 years earlier. In Section 2 we present some basic facts about frames and pseudo-inverses. Most of the material has been published elsewhere (see e.g. [B], [BR]). In Section 3 we link the two topics. Given a frame ff i g in a Hilbert space 3 The author would like to thank the Carlsberg Foundation and the Danish Research Academy for nancial support. 1
H we show some important relations between the inverse frame operator and the pseudo-inverse of T : l 2 (I)! H ; T fc i g := c i f i : The results will be applied to the following problem: if R is an operator on H, is there a convenient way to calculate the coecients in the frame expansion of Rf from the coecients in the expansion of f? Section 4 is motivated by the fact that it often is possible to make well- behaving decompositions with respect to sequences ff i g which do not satisfy the upper frame condition. In the literature, this situation has been treated by introduction of weights (scaling factors); we shall give a completely dierent approach, where the main tool is the theory of pseudo-inverses of unbounded operators. Acknowledgement: The author is indebted to Hans G. Feichtinger, who introduced him to the pseudo-inverse of a matrix. Some of the results in Section 3 have been proposed in nite dimensional spaces by Feichtinger and then proved by the author in the setting of a Hilbert space. 2 Basic facts. Throughout the paper the term "Hilbert space" means an innite dimensional separable Hilbert space, with the inner product linear in the rst entry. Let us begin with a short review over the needed facts about pseudo-inverses. Let H and K be Hilbert spaces and T : K! H a bounded linear operator with closed range R T. Denote the null space by N T : T restricted to N? T (the orthogonal complement of the null space) is an injective operator. Let us consider it as an operator into R T, i.e. dene ~T := T jnt? : N T?! R T : ~T is bijective and has a bounded inverse ~T 01 : R T! N T? : 2
The pseudo-inverse of T is now dened as the unique extension T y of (T ~ ) 01? to an operator on H with the property N T y = R T : Another word for pseudo-inverse is Moore-Penrose-inverse. Let us use the abbreviation PI from now. The PI gives us the solution to a very interesting problem, see ( [BR] Cor. 1.1): Theorem 2.1: Let f 2 R T. The equation T x = f has exactly one solution with minimal norm; this solution is T y f: Remark: We have introduced the PI such that it ts to our applications in Section 3. The PI can also be dened for operators without closed range and Theorem 2.1 can be extended to a result telling that T y f solves a certain approximation problem for a general f 2 H; we don't need these facts. In Section 4 we shall introduce the PI for densely dened operators with closed range. In general, the product rule does not apply to the PI. The main result in [BO] is Theorem 2.2: Let V be a bounded operator from H into K and U a bounded operator from K into H; suppose that R U and R V are closed. Then (U V ) y = V y U y i (i) R U V is closed (ii) R U 3 is invariant under V V 3 and (iii) R U 3 \ N V 3 is invariant under U 3 U. In Section 3 we shall use a consequence of Theorem 2.2: Corollary 2.3: Let U be a bounded operator with closed range. Then (U 3 U) y = U y U 3y : Proof: By the closed range theorem ([R] Th. 4.14) also R U 3 is closed. Since 3
R U? = N U 3, R U 3 U = U 3 UH = U 3 (UH + (UH)? ) = U 3 H: It is also easy to check that (ii) and (iii) are satised. We need one more fact about the PI ([BR ] Th. 1.6): Lemma 2.4: (T y ) 3 = (T 3 ) y : A family ff i g of vectors in H is called a Bessel sequence if j < f; f i > j 2 < 1 ; 8f 2 H: If ff i g is a Bessel sequence, then there exists a constant B > 0 such that j < f; f i > j 2 Bkfk 2 ; 8f 2 H: P Furthermore, c if i mapping converges unconditionally for all fc i g 2 l 2 (I); the T : fc i g! c i f i is bounded from l 2 (I) into H, with kt k p B: Composing T with the adjoint operator T 3 : f 70! f< f; f i >g we get the so called frame operator S : H! H ; Sf = < f; f i > f i : S is a bounded positive operator with ksk B: The Bessel sequence ff i g is called a frame if 9A > 0 : Akfk 2 j < f; f i > j 2 ; 8f 2 H: 4
Any pair of positive numbers A and B such that Akfk 2 j < f; f i > j 2 Bkfk 2 ; 8f 2 H will be called a set of frame bounds. Sometimes we shall use the term "optimal bounds"; these are A = inf kfk=1 j < f; f i > j 2 and B = sup kfk=1 j < f; f i > j 2 : A frame provides us with a decomposition of H. In fact, S is a bijection on H, so f = SS 01 f = < S 01 f; f i > f i = < f; S 01 f i > f i ; 8f 2 H: Using elementary functional analysis an equivalent characterization of frames has been proved in [C] Th. 2.5: Theorem 2.5: A family ff i g frame operator is well dened and surjective. of vectors in H is a frame if and only if the Here the term "well dened" means that the partial sums corresponding to some enumeration of the elements in I converges. In fact it is possible to be more general. Using S = T T 3 we obtain R S = SH = T T 3 H = T (T 3 H + (T 3 H)? ) since R? T 3 = N T : The closed range theorem tells us that R T 3 is closed if R T is closed; in this case T 3 H + (T 3 H)? = H and R S = R T. In particular the condition "S surjective" can be replaced by "T surjective". We shall give a completely dierent proof of this version of Theorem 2.5 in Section 4. The frame expansion has a certain minimal property (see [DS] or [HW]): 5
P Lemma 2.6 Let ff i g be a frame and f an element in H. If f = c if i then jc i j 2 = j < f; S 01 f i > j 2 + We shall use this result in Section 3. 3 01 and y j < f; S 01 f i > 0c i j 2 : Let H be a Hilbert space and V H a closed subspace with frame ff i g. The corresponding frame operator S will be considered as an operator from V into V. Our aim is to study the relation between S 01 and T y, where T : l 2 (I)! H ; T fc i g = c i f i : Theorem 3.1: T y f = f< f; S 01 f i >g ; 8f 2 H: Proof: Let P denote the orthogonal projection of H onto V and let f 2 H. Consider the equation T fc i g = P f, i.e. (1) c i f i = P f: By Theorem 2.1 (1) has a unique solution with minimal norm, namely fc i g = T y P f: By denition, T y is zero on R? T = V? ; therefore fc i g = T y P f + T y (I 0 P )f = T y f: But P f 2 V, so the usual frame decomposition applies: P f = < P f; S 01 f i > f i = < f; S 01 f i > f i : 6
Lemma 2.6 tells us that the sequence f< f; S 01 f i >g solves the same problem; therefore T y f = f< f; S 01 f i >g : Remark: We can express the result in the following way: if we know fs 01 f i g, then we can nd T y f. It is not dicult to prove that the opposite is true. If 3 = f i;k g is the matrix for T y with respect to an orthonormal basis fa i g for l 2 (I) and an orthonormal basis fe i g for H, then S 01 f i = k2i i;k e k ; 8f 2 H: For convenience we use the index set I = N in the next result. Still ff i g 1 i=1 is a frame for the closed subspace V H: We have already used that any f 2 V has a unique minimal norm expansion in terms of ff i g 1 i=1, namely the frame decomposition f = 1 i=1 < f; S 01 f i > f i : Let now R be a bounded operator on H P, mapping V into V. The minimal norm decomposition of Rf is Rf = i=1 < Rf; S01 f i > f i : It is not 1 dicult to nd a useful relation between the sequences f< f; S 01 f i >g and f< Rf; S 01 f i >g: < Rf; S 01 f i >=< f; R 3 S 01 f i >=< P 1 j=1 < f; S01 f j > f j ; R 3 S 01 f i > = P 1 j=1 < f; S01 f j >< Rf j ; S 01 f i > : Let 5 denote the innite matrix where the ij'th entry is i;j =< Rf j ; S 01 f i >. Our calculation shows that f< Rf; S 01 f i >g 1 i=1 = 5f< f; S01 f i >g 1 i=1 : 7
Lemma 3.2: 5 denes a bounded operator on l 2 (N). Proof: Let fc j g 1 j=1 2 l 2 (N): Let A,B be some frame bounds; then P 1 i=1 jp 1 j=1 i;jc j j 2 = P 1 i=1 jp 1 j=1 < Rf j ; S 01 f i > c j j 2 = P 1 i=1 j < RP 1 j=1 c j f j ; S 01 f i > j 2 1 A krp 1 j=1 c jf j k 2 1 A krk2 1 k P 1 j=1 c j f j k 2 B A krk2 1 P 1 j=1 jc jj 2 : (We have used that fs 01 f i g is a frame with upper bound 1 ): This means A that 5fc j g is an element in l 2 (N): q Clearly, 5 is bounded with norm less B than or equal to 1 krk: A Let us mention two interesting special cases: (i) For R = S; i;j =< Sf j ; S 01 f i >=< f j ; f i > : That is, 5 is just the transpose of the Gram matrix for ff i g 1 i=1 : (ii) For R = S 01 ; i;j =< S 01 f j ; S 01 f i > : That is, 5 is the transpose of the Gram matrix corresponding to the dual frame fs 01 f i g 1 i=1 : Using the remark after Theorem 3.1 we obtain another expression for 5: 1 i;j =< Rf j ; S 01 f i >=< Rf j ; i;k e k >= i;k < Rf j ; e k > : k=1 k=1 That is, 5 is the matrix product of 3 and the matrix 0 = f k;j g, where k;j =< Rf j ; e k > : 1 8
Again, in the special case R = S 01, k;j =< S 01 f j ; e k >=< 1 j;l e l ; e k >= j;k ; l=1 that is, 5 = 33 3 : According to the well known rules for calculations with matrix representations for operators, 33 3 is also the matrix for the operator T y T y3 = T y T 3y = (T 3 T ) y : T 3 T is just the transpose of the Gram matrix for ff i g 1 i=1, so f< f i ; f j >g y = f< S 01 f i ; S 01 f j >g: In the applications of frames (wavelet theory, irregular sampling of bandlimited functions) it is very important to have good estimates for the optimal frame bounds. The reason is that they play a decisive role for the speed of convergence for some reconstruction algorithms. For further details, see [FG]. The optimal frame bounds can be expressed in terms of kt k and kt y k. Our result is almost a corollary of the next lemma, which exhibits the relation between the optimal bounds for the frame ff i g and the dual frame fs 01 f i g : We leave the proof to the reader. Lemma 3.3: Let A; B be the optimal bounds for the frame ff i g : The dual frame fs 01 f i g has the optimal bounds 1 B and 1 A : Proposition 3.4: The optimal frame bounds for ff i g are A = ks 01 k 01 = kt y k 02 and B = ksk = kt k 2 : Proof: By denition, B = sup kfk=1 j < f; f i > j 2 = sup kfk=1 < Sf; f >= ksk: Using this result on the dual frame (which has the frame operator S 01 and the optimal upper bound 1 A ) we obtain that 1 A = ks01 k: 9
Since S = T T 3, also ksk = kt T 3 k = kt k 2, By Theorem 3.1, T y f = f< f; S 01 f i >g ; 8f 2 H: Therefore kt y fk 2 = j < f; S 01 f i > j 2 1 A kfk2 ; 8f 2 H: By Lemma 3.3, 1 is the smallest possible constant with this property; this A means that kt y k 2 = 1 : A Remark: As shown in [TB], the optimal frame bounds can also be expressed in terms of the norms of the so called frame correlation operator and it's pseudo-inverse. Teolis and Benedetto also shows how results of this type can be used in applications. 4 Decomposition via unbounded operators. The frame conditions are very strong, which implies that it is very convenient to work with the theory: all relevant operators are bounded, the frame decompositions converge unconditionally, and so on. Our aim here is to take the rst step towards a more general decomposition theory. We shall study sequences which are not Bessel sequences and show that it sometimes is possible to make a "frame-like decomposition." Let us explain why it is important. The motivation behind frames is that a frame ff i g can be overcomplete, so we have freedom in the choice of the expansion coecients. Compare with the situation of an orthornormal basis, where every element in H has a unique decomposition in terms of the basis. But if a family ff i g has too much "redundancy", then it may fail to be a Bessel sequence, i.e. it can not build a frame. Without going into details we mention, that for example in the context of irregular sampling it can happen if there are too many sampling points. So it is not only of theoretically interest that the theory under some technically conditions can be generalized to cover such a situation. The starting point is the observation that the pseudo-inverse can be de- ned for any closed densely dened operator. In general, the pseudo-inverse 10
of such an operator is not uniquely dened. But we shall only work with operators with closed range; for such operators there is no ambiguity. Let again H and K be Hilbert spaces and T : K! H a densely dened closed operator with closed range. By [BR] Lemma 1.1 the pseudo-inverse T y of T can be dened as follows: Denition: T y is the uniquely dened linear operator from H into K satisfying (i)-(iii) below: (i) N T y = R T? : (ii) R T y = N T? (iii) T T y f = f; 8f 2 R T : Theorem 2.1 remains true in this more general setting. By [BR] Corollary 1.2, T y is bounded. Let ff i g be a family of vectors in H. We do not assume ff i g to be a Bessel sequence, so even for fc i g 2 l 2 (I) we have to explain the meaning of expressions like P c if i : Fix once and for all an enumeration of the elements in I. If the corresponding sequence of partial sums converges, we shall denote the limit by P c if i : Corresponding to the family ff i g we dene the - maybe unbounded - operator T : D(T ) l 2 (I)! H by T fc i g := c i f i ; D(T ) = ffc i g 2 l 2 (I) j c i f i converges:g: The nite sequences are dense in l 2 (I) and contained in D(T ); thus T is densely dened. Our rst result contains the generalized frame decomposition. Furthermore it shows that the existence of the lower frame bound is preserved also in this more general setting: Theorem 4.1: Suppose that T is closed and surjective. Then 11
(i) There exists a Bessel sequence fg i g in H such that f = < f; g i > f i ; 8f 2 H: (ii) 1 kt y k 2 1 kfk 2 P j < f; f i > j 2 ; 8f 2 H: Proof: Any f 2 H can be decomposed as f = T T y f = (T y f) i f i ; and kt y fk 2 = j(t y f) i j 2 kt y k 2 1 kfk 2 : In particular, f 70! (T y f) i is for any i 2 I a continuous linear functional on H, so for any i 2 I there exists a uniquely determined element g i 2 H such that Furthermore, j < f; g i > j 2 = (T y f) i =< f; g i >; 8f 2 H: j(t y f) i j 2 kt y k 2 1 kfk 2 ; 8f 2 H: Part (ii) is a consequence of the same two facts. Given f 2 H, kfk 4 = j < f; f > j 2 = j P (T y f) i < f i ; f > j 2 P j(t y f) i j 2 P j < f; f i > j 2 kt y k 2 1 kfk 2 P j < f; f i > j 2 : According to the proof of Theorem 4.1, the sequence fg i g satisfying T y f = f< f; g i >g ; 8f 2 H can be used in (i). With this choice we obtain a generalization of Lemma 2.6: 12
Proposition 4.2: Suppose that T is closed and surjective,and let f 2 H: If f = P c if i, then jc i j 2 = j < f; g i > j 2 + jc i 0 < f; g i > j 2 : Proof: If f = P c if i, then P [c i0 < f; g i >]f i = 0; that is, fc i 0 < f; g i >g 2 N T = R T y? : By denition, Therefore f< f; g i >g = T y f 2 R T y: kfc i 0 < f; g i >gk 2 + kf< f; g i >gk 2 = kfc i gk 2 : If ff i g is a frame, then fg i g is the dual frame. In the general case, let us call fg i g the dual family of ff i g. Theorem 4.1 contains interesting results about frames: Corollary 4.3: A Bessel sequence ff i g is a frame if and only if T is surjective. Proof: If ff i g is a frame, S = T T 3 is surjective. In general, when ff i g is a Bessel sequence T is a bounded operator dened on all of l 2 (I) and therefore closed. If also R T = H, Theorem 4.1 (ii) tells us that the lower frame condition is satised. Corollary 4.4: For any family ff i g, (i) and (ii) below are equivalent: (i) ff i g is a frame for spanff i g. (ii) T is bounded and dened on all of l 2 (I) and R T is closed. 13
Proof: (i) ) (ii) is clear. (ii) ) (i): If T is well dened on l 2 (I), ff i g is a Bessel sequence. Clearly, spanff i g R T spanff i g so if R T is closed R T = spanff i g : Now the result follows from Corollary 4.3 applied to ff i g considered as a sequence in the Hilbert space spanff i g : A similar statement (using T 3 instead of T ) can be found in [TB, Theorem 3.3]. The two results were found at the same time, but independently of each other. The rest of the paper is devoted to a study of the rather abstract condition "T closed." The condition is satised if T is invertible and the inverse bounded. In general, the situation is anything but trivial. Given a Bessel sequence ff i g the bounded operator U : H! l 2 (I); U f = f< f; f i >g ; is the adjoint of T, i.e. T 3 = U; U 3 = T: For general sequences the situation is more complicated. Given a sequence ff i g in H we dene the operator U : D(U) H! l 2 (I) by U f := f< f; f i >g ; D(U) = ff 2 H j j < f; f i > j 2 < 1g: Proposition 4.5: Suppose that P j < f j; f i > j 2 < 1; 8j 2 I: Then U is densely dened. We leave the proof to the reader. Our interest for U comes from the fact that U 3 is closed if U is densely dened ([R]Th. 13.9). 14
Proposition 4.6: Suppose that U is densely dened. Then T U 3 : Proof: By denition, D(U 3 ) = ffc i g 2 l 2 (I) j f 70!< U f; fc i g > l 2(I) is continuous on D(U)g P = ffc i g 2 l 2 (I) j c i < f; f i > is continuous on D(U)g Now suppose that fc i g 2 D(T ), i.e. that P c if i converges.then j c i < f; f i > j 2 = j < f; c i f i > j 2 kfk 1 k c i f i k; 8f 2 H: Therefore f 70! P c i < f; f i > is continuous on H, so D(T ) D(U 3 ): Furthermore, for f 2 D(U) and fc i g 2 D(T ), < U f; fc i g >= c i < f; f i >= j < f; i.e. U 3 fc i g = T fc i g for fc i g 2 D(T ): Corollary 4.7: T is closed if c i f i > j =< f; T fc i g > ffc i g 2 l 2 (I) j f 70! P c i < f; f i > is continuous on D(U)g = ffc i g 2 l 2 (I) j P c if i converges:g: The condition in Corollary 4.7. is not always satised. Here is an example: Let fe i g 1 i=1 be an orthonormal basis for H. Dene Clearly, f1 = e1; f i = i(e i 0 e i01 ); i 2: 1 j < f j ; f i > j 2 < 1; i=1 8j 15
so Proposition 4.5 tells that U is densely dened. Let c i = 1 ; i 2 N: Then i n so fc i g is not in D(T ): But c i f i = e1 + i=1 n i=1 n (e i 0 e i01 ) = e n i=2 c i < f; f i >=< f; e n > 8f 2 H; which implies that P 1 i=1 c i < f; f i >= 0; 8f 2 H: In particular fc i g is in D(U 3 ). References: [B] Beutler, F.J.: The operator theory of the pseudo-inverse. J. Math. Anal. Appl. 10 (1965), p.451-493. [BR] Beutler, F.J. and Root, W. L.:The operator pseudo-inverse in control and systems identications. In "Generalized inverses and applications". Ed. M. Zuhair Nashed. Academic Press 1976. [Bo] Bouldin, R.: The pseudo-inverse of a product. 24 (1973), p.489-495. SIAM J. Appl. Math. [C] Christensen, O: Frames and the projection method. Applied and Computational Harmonic Analysis 1 (1993), p.50-53. [D] Daubechies, I.: The wavelet transform, time-frequency localization and signal analysis. IEEE Trans. Inform. Theory 36 (1990), p.961-1005. [DGM] Daubechies, I., Grossmann, A. and Meyer, Y.: Painless non orthogonal expansions. J.Math. Phys. 27 (1986), p.1271-1283. 16
[DS] Dun, R. and Schaeer, A.: A class of non harmonic Fourier series. Trans. Amer. Math. Soc. 72 (1952), p.341-366. [F] Feichtinger, H.G.: Pseudo-inverse matrix methods for signal reconstruction from partial data. SPIE-Conf., Visual Comm. and Image Processing; Boston, Nov.1991. Int.Soc.Opt.Eng., p. 766{772. [FG] Feichtinger, H. G. and Grochenig, K.:"Theory and practice of irregular sampling." Appeared in "Wavelets: Mathematics and Applications." Eds. Benedetto, J. and Frazier, M.. CRC Press 1993. [H] Heuser,H.: Functional analysis. John Wiley & Sons 1982. [HW] Heil, C. and Walnut, D.F.: Continuous and discrete wavelet transforms. SIAM Review 31 (1989), p.628-666. [R] Rudin, W.: Functional analysis. McGraw-Hill, 1973 [TB] Teolis, A. and Benedetto, J.: Local Frames. SPIE Math. Imaging. 2034 (1993) p.310-321. Ole Christensen Mathematical Institute The Technical University of Denmark Building 303 2800 Lyngby Denmark. E-mail: OLECHR@MAT.DTH.DK 17