arxiv: v1 [stat.ml] 10 Mar 2016

Transcription

1 Global and Local Uncertainty Princiles for Signals on Grahs Nathanael Perraudin, Benjamin Ricaud, David I Shuman, and Pierre Vandergheynst March, 206 arxiv: v [stat.ml] 0 Mar 206 Abstract Uncertainty rinciles such as Heisenberg s rovide limits on the time-frequency concentration of a signal, and constitute an imortant theoretical tool for designing and evaluating linear signal transforms. Generalizations of such rinciles to the grah setting can inform dictionary design for grah signals, lead to algorithms for reconstructing missing information from grah signals via sarse reresentations, and yield new grah analysis tools. While revious work has focused on generalizing notions of sreads of a grah signal in the vertex and grah sectral domains, our aroach is to generalize the methods of Lieb in order to develo uncertainty rinciles that rovide limits on the concentration of the analysis coefficients of any grah signal under a dictionary transform whose atoms are jointly localized in the vertex and grah sectral domains. One challenge we highlight is that due to the inhomogeneity of the underlying grah data domain, the local structure in a single small region of the grah can drastically affect the uncertainty bounds for signals concentrated in different regions of the grah, limiting the information rovided by global uncertainty rinciles. Accordingly, we suggest a new way to incororate a notion of locality, and develo local uncertainty rinciles that bound the concentration of the analysis coefficients of each atom of a localized grah sectral filter frame in terms of quantities that deend on the local structure of the grah around the center vertex of the given atom. Finally, we demonstrate how our roosed local uncertainty measures can imrove the random samling of grah signals. Index terms Signal rocessing on grahs, uncertainty rincile, local uncertainty, time-frequency analysis, localization, concentration bound, non-uniform random samling Introduction The major research thrust to date in the emerging area of signal rocessing on grahs [, 2] has been to design multiscale wavelet and vertex-frequency transforms [3]-[24]. Objectives of these transforms are to sarsely reresent different classes of grah signals and/or efficiently reveal relevant structural roerties of high-dimensional data on grahs. As we move forward, it is imortant to both test these transforms on myriad alications, as well as to develo additional theory to hel answer the question of which transforms are best suited to which tyes of data. Uncertainty rinciles such as the ones resented in [25]-[30] are an imortant tool in designing and evaluating linear transforms for rocessing classical signals such as audio signals, time series, and images residing on Euclidean domains. It is desirable that the dictionary atoms are jointly localized in time and frequency, and uncertainty rinciles characterize the resolution tradeoff between these two domains. Moreover, while the uncertainty rincile is [often] used to show that certain things are imossible, Donoho and Stark [25] resent examles where the generalized uncertainty rincile shows something unexected is ossible; secifically, the recovery of a signal or image desite significant amounts of missing information. In articular, uncertainty rinciles can rovide guarantees that if a signal has a sarse decomosition in a dictionary of incoherent atoms, this is indeed a unique reresentation that can be recovered via otimization [26, 27]. This idea underlies the recent wave of sarse signal rocessing techniques, with alications such as denoising, source searation, inainting, and comressive sensing. While there is still limited theory showing that different mathematical classes of grah signals are sarsely reresented by the recently roosed transforms (see [3] for one reliminary work along these lines), there is far more emirical work showing the otential of these transforms to sarsely reresent grah signals in various alications. Many of the multiscale transforms designed for grah signals attemt to leverage intuition from signal rocessing techniques designed for signals on Euclidean data domains by generalizing fundamental oerators and transforms to the grah setting (e.g.,

2 by checking that they corresond on a ring grah). While some intuition, such as the notion of filtering with a Fourier basis of functions that oscillate at different rates (see, e.g., []) carries over to the grah setting, the irregular structure of the grah domain often restricts our ability to generalize ideas. One rime examle is the lack of a shift-invariant notion of translation of a grah signal. As shown in [32, 33] and discussed in [23, Section 3.2], the concentration of the Fourier basis functions is another examle where the intuition does not carry over directly. Comlex exonentials, the basis functions for the classical Fourier transform, have global suort across the real line. On the other hand, the eigenvectors of the combinatorial or normalized grah Lalacians, which are most commonly used as the basis functions for a grah Fourier transform, are sometimes localized to small regions of the grah. Because the incoherence between the Fourier basis functions and the standard normal basis underlies many uncertainty rinciles, we demonstrate this issue with a short examle. Motivating Examle (Part I: Lalacian eigenvector localization). Let us consider the two manifolds (surfaces) embedded in R 3 and shown in the first row of Figure. The first one is a flat square. The second is identical excet for the center where it contains a sike. We samle both of these manifolds uniformly across the x-y lane and create a grah by connecting the 8 nearest neighbors with weights deending on the distance (W ij = e dij/σ ). The energy of each Lalacian eigenvector of the grah arising from the first manifold is not concentrated on any articular vertex; i.e., max i,l u l (i) <<, where u l is the eigenvector associated with eigenvalue λ l. However, the grah arising from the second manifold does have a few eigenvectors, such as eigenvector 3 shown in the middle row Figure, whose energy is highly concentrated on the region of the sike; i.e: max i,l u l (i). Yet, the Lalacian eigenvectors of this second grah whose energy resides rimarily on the flatter regions of the manifold, such as eigenvector 7 shown in the bottom row of Figure, are not too concentrated on any single vertex. Rather, they more closely resemble some of the Lalacian eigenvectors of the grah arising from the first manifold. Below we discuss three different families of uncertainty rinciles, and their extensions to the grah setting, both in rior work and in this contribution. The first family of uncertainty rinciles measure the sreading around some reference oint, usually the mean osition of the energy contained in the signal. The well-known Heisenberg uncertainty rincile [34, 35] belongs to this family. It views the modulus square of the signal in both the time and Fourier domains as energy robability density functions, and takes the variance of those energy distributions as measures of the sreading in each domain. The uncertainty rincile states that the roduct of variances in the time and in the Fourier domains cannot be arbitrarily small. The generalization of this uncertainty rincile to the grah setting is comlex since there does not exist a simle formula for the mean value or the variance of grah signals, in either the vertex or the grah sectral domains. For unweighted grahs, Agaskar and Lu [36, 37, 38] also view the square modulus of the signal in the vertex domain as an energy robability density function and use the geodesic grah distance (shortest number of hos) to define the sread of a grah signal around a given center vertex. For the sread of a signal f in the grah sectral domain, Agaskar and Lu use the normalized variation f Lf, f 2 2 which catures the smoothness of a signal. They then secify uncertainty curves that characterize the tradeoff between the smoothness of a grah signal and its localization in the vertex domain. This idea is generalized to weighted grahs in [39]. As ointed out in [38], the tradeoff between smoothness and localization in the vertex domain is intuitive as a signal that is smooth with resect to the grah toology cannot feature values that decay too quickly from the eak value. However, as shown in Figure (and subsequent examles in Table ), grah signals can indeed be simultaneously highly localized or concentrated in both the vertex domain and the grah sectral domain. This discreancy is because the normalized variation used as the sectral sread in [38] is one method to measure the sread of the sectral reresentation around the eigenvalue 0, rather than around some mean of that signal in the grah sectral domain. In fact, using the notion of sectral sread resented in [38], the grah signal with the highest sectral sread on a grah G is the grah Lalacian eigenvector associated with the highest eigenvalue. The grah sectral reresentation of that signal is a Kronecker delta whose energy is comletely localized at a single eigenvalue. One might argue that its sread should in fact be zero. So, in summary, while there does exist a tradeoff between the smoothness of a grah signal and its localization around any given center vertex in the vertex domain, the classical idea that a signal cannot be simultaneously localized in the time and frequency domains does not always carry over to the grah setting. While certainly an interesting avenue for continued investigation, we do not discuss uncertainty rinciles based on sreads in the vertex and grah sectral domains any further in this aer. 2

3 Manifold Manifold 2 Eigenvector 5 (Manifold ) Eigenvector 3 (Manifold 2) Eigenvector 3 (Manifold ) Eigenvector 7 (Manifold 2) Figure : Concentration of grah Lalacian eigenvectors. We discretize two different manifolds by samling uniformly across the x-y lane. Due its bumy central art, the grah arising from manifold 2 has a grah Lalacian eigenvector (shown in the middle row of the right column) that is highly concentrated in both the vertex and grah sectral domains. However, the eigenvectors of this grah whose energy rimarily resides in the flatter arts of the manifold (such as the one shown in the bottom row of the right column) are less concentrated, and some closely resemble the Lalacian eigenvectors of the grah arising from the flat manifold (such as the corresonding eigenvector shown in the bottom row of the left column. The second family of uncertainty rinciles involve the absolute sarsity or concentration of a signal. The key quantities are tyically either suort measures counting the number of non-zero elements, or concentration measures, such as l - norms. An imortant distinction is that these sarsity and concentration measures are not localization measures. They can give the same values for different signals, indeendent of whether the redominant signal comonents are clustered in a small region of the vertex domain or sread across different regions of the grah. An examle of a recent work from the grah signal rocessing literature that falls into this family is [40], in which Tsitsvero et al. roose an uncertainty rincile that characterizes how jointly concentrated grah signals can be in the vertex and sectral domains. Generalizing rolate sheroidal wave functions [4], their notion of concentration is based on the ercentage of energy of a grah signal that is concentrated on a given set of vertices in the vertex domain and a given set of frequencies in the grah sectral domain. Since we can interret signals defined on grahs as finite dimensional vectors with well-defined l -norms, we can also 3

4 aly directly the results of existing uncertainty rinciles for finite dimensional signals. As one examle, the Elad- Bruckstein uncertainty rincile of [27] states that if α and β are the coefficients of a vector f R N in two different orthonormal bases, then α 0 + β 0 2 α 0 β 0 µ, () where µ is the maximum magnitude of the inner roduct between any vector in the first basis with any vector in the second basis. In Section 3., we aly () to grah signals by taking one basis to be the canonical basis of Kronecker delta functions in the grah vertex domain and the other to be a Fourier basis of grah Lalacian eigenvectors. We also aly other such finite dimensional uncertainty rinciles from [30], [34], and [42] to the grah setting. In Section 3.2, we adat the Hausdorff-Young inequality [43, Section IX.4], a classical result for infinite dimensional signals, to the grah setting. These results tyically deend on the mutual coherence between the grah Lalacian eigenvectors and the canonical basis of deltas. For the secial case of shift-invariant grahs with circulant grah Lalacians [44, Section 5.], such as ring grahs, these bases are incoherent, and we can attain meaningful uncertainty bounds. However, for less homogeneous grahs (e.g., a grah with a vertex with a much higher or lower degree than other vertices), the two bases can be more coherent, leading to weaker bounds. Moreover, as we discuss in Section 2, the bounds are global bounds, so even if the majority of a grah is for examle very homogenous, inhomogeneity in one small area can revent the result from informing the behavior of grah signals across the rest of the grah. The third family of uncertainty rinciles characterize a single joint reresentation of time and frequency. The shorttime Fourier transform (STFT) is an examle of a time-frequency reresentation that rojects a function f onto a set of translated and modulated coies of a function g. Usually, g is a function localized in the time-frequency lane, for examle a Gaussian, vanishing away from some known reference oint in the joint time and frequency domain. Hence this transformation reveals local roerties in time and frequency of f by searating the time-frequency domain into regions where the translated and modulated coies of g are localized. This reresentation obeys an uncertainty rincile: the STFT coefficients cannot be arbitrarily concentrated. This can be shown by estimating the different l -norms of this reresentation (note that the concentration measures of the second family of uncertainty rinciles are used). For examle, Lieb [45] roves a concentration bound on the ambiguity function (e.g., the STFT coefficients of the STFT atoms). Lieb s aroach is more general than the Heisenberg uncertainty rincile, because it handles the case where the signal is concentrated around multile different oints (see, e.g., the signal f 3 in Figure 2). In Section 5, we generalize Lieb s uncertainty rincile to the grah setting to rovide uer bounds on the concentration of the transform coefficients of any grah signal under (i) any frame of dictionary atoms, and (ii) a secial class of dictionaries called localized sectral grah filter frames, whose atoms are of the form T i g k, where T i is a localization oerator that centers on vertex i a attern described in the grah sectral domain by the kernel ĝ k. While the second family of uncertainty rinciles above yields global uncertainty rinciles, we can generalize the third family to the grah setting in a way that yields local uncertainty rinciles. In the classical Euclidean setting, the underlying domain is homogenous, and thus uncertainty rinciles aly to all signals equally, regardless of where on the real line they are concentrated. However, in the grah setting, the underlying domain is irregular, and a change in the grah structure in a single small region of the grah can drastically affect the uncertainty bounds. For instance, the second family of uncertainty rinciles all deend on the coherence between the grah Lalacian eigenvectors and the standard normal basis of Kronecker deltas, which is a global quantity in the sense that it incororates local behavior from all regions of the grah. To see how this can limit the usefulness of such global uncertainty rinciles, we return to the motivating examle from above. Motivating Examle (Part II: Global versus local uncertainty rinciles). In Section 3., we show that a direct alication of a result from [30] to the grah setting yields the following uncertainty relationshi, which falls into the second family described above, for any signal f R N : ) ( ) ( f 2 ˆf 2 f ˆf max u l (i). (2) i,l 4

5 Each fraction in the left-hand side of (2) is a measure of concentration that lies in the interval [ N, ] (N is the number of vertices), and the coherence between the grah Lalacian eigenvectors and the Kronecker deltas on the right-hand side lies in the same interval. On the grah arising from manifold, the coherence is close to N, and (2) yields a meaningful uncertainty rincile. However, on the grah arising from manifold 2, the coherence is close to due to the localized eigenvector 3 in Figure, (2) is trivially true for any signal in R n from the roerties of vector norms, and thus the uncertainty rincile is not articularly useful. Nevertheless, far away from the sike, signals should behave similarly on manifold 2 to how they behave on manifold. Part of the issue here is that the uncertainty relationshi holds for any grah signal f, even those concentrated on the sike, which we know can be jointly localized in both the vertex and grah sectral domains. An alternative aroach is to develo a local uncertainty rincile that characterizes the uncertainty in different regions of the grah on a searate basis. Then, if the energy of a given signal is concentrated on a more homogeneous art of the grah, the concentration bounds will be tighter. In Section 6, we generalize the aroach of Lieb to build a local uncertainty rincile that bounds the concentration of the analysis coefficients of each atom of a localized grah sectral filter frame in terms of quantities that deend on the local structure of the grah around the center vertex of the given atom. Thus, atoms localized to different regions of the grah feature different concentration bounds. Such local uncertainty rinciles also have constructive alications, and we conclude with an examle of non-uniform samling for grah inainting, where the varying uncertainty levels across the grah suggest a strategy of samling more densely in areas of higher uncertainty. For examle, if we were to take M measurements of a smooth signal on manifold 2 in Figure, this method would lead to a higher robability of samling signal values near the sike, and a lower robability of samling signal values in the more homogenous flat arts of the manifold, where reconstruction of the missing signal values is inherently easier. 2 Notation and grah signal concentration In this section, we introduce some notation and illustrate further how certain intuition from signal rocessing on Euclidean saces does not carry over to the grah setting. 2. Notation Throughout the aer, we consider signals residing on an undirected, connected, and weighted grah G = {V, E, W}, where V is a finite set of N vertices ( V = N), E is a finite set of edges, and W is the weight or adjacency matrix. The entry W ij of W reresents the weight of an edge connecting vertices i and j. A grah signal f : V C is a function assigning one value to each vertex. Such a signal f can be written as a vector of size N with the n th comonent reresenting the signal value at the n th vertex. The generalization of Fourier analysis to the grah setting requires a grah Fourier basis {u l } l {0,,...,N }. The most commonly used grah Fourier bases are the eigenvectors of the combinatorial (or non-normalized) grah Lalacian, which is defined as L = D W, where D is the diagonal degree matrix with diagonal entries D ii = N j= W ij, and i V, or the eigenvectors of the normalized grah Lalacian L = D 2 LD 2. However, the eigenbases (or Jordan eigenbases) of other matrices such as the adjacency matrix have also been used as grah Fourier bases [46, 2]. All of our results in this aer hold for any choice of the grah Fourier basis. For concreteness, we use the combinatorial Lalacian, which has a comlete set of orthonormal eigenvectors {u l } l {0,,...,N } associated with the real eigenvalues 0 = λ 0 < λ λ 2... λ N = λ max. The grah Fourier transform ˆf C N of a function f C N defined on a grah G is the rojection of the signal onto the orthonormal grah Fourier basis {u l } l=0,,...,n, which we take to be the eigenvectors of the grah Lalacian associated with G: N ˆf(λ l ) = f, u l = f(n)u l (n), l {0,,..., N }. (3) n= See for examle [47] for more details on sectral grah theory, and [] for more details on signal rocessing on grahs. 5

6 2.2 Concentration measures In order to discuss uncertainty rinciles, we must first introduce some concentration/sarsity measures. Throughout the aer, we use the terms sarsity and concentration somewhat interchangeably, but we reserve the term sread to describe the sread of a function around some mean or center oint, as discussed in the first family of uncertainty rinciles in Section. The first concentration measure is the suort measure of f, denoted f 0, which counts the number of non-zero elements of f. The second concentration measure is the Shannon entroy, which is used often in information theory and hysics: H(f) = n f(n) 2 ln f(n) 2, where the variable n has values in {, 2,..., N} for functions on grahs and in {0,,..., N } in the grah Fourier reresentation. Another class of concentration measures is the l -norms, with [, ]. For 2, the sarsity of f may be measured using the following quantity: f 2 f, if 2 s (f) =. For any vector f C N and any [, ], s (f) f f 2, if 2 < [N ] 2,. If s (f) is high (close to ), then f is sarse, and if s (f) is low, then f is not concentrated. Figure 2 uses some basic signals to illustrate this notion of concentration, for different values of. In addition to sarsity, one can also relate l -norms to the Shannon entroy via Renyi entroies (see, e.g., [48, 49] for more details). s (f ) s (f 2 ) s (f 3 ) s (f 4 ) Figure 2: The concentration s ( ) of four different examle signals (all with 2-norm equal to ), for various values of. Note that the osition of the signal coefficients does not matter for this concentration measure. Different values of lead to different notions of concentration; for examle, f 2 is more concentrated than f 3 if = (it has a larger maximum absolute value), but less concentrated if =. 2.3 Concentration of the grah Lalacian eigenvectors The sectrum of the grah Lalacian relaces the frequencies as coordinates in the Fourier domain. For the secial case of shiftinvariant grahs with circulant grah Lalacians [44, Section 5.], the Fourier eigenvectors can still be viewed as ure oscillations. However, for more general grahs (i.e., all but the most highly structured), the oscillatory behavior of the Fourier eigenvectors must be interreted more broadly. For examle, [, Fig. 3] dislays the number of zero crossings of each eigenvector; that is, 6

7 for each eigenvector, the number of airs of connected vertices where the signs of the values of the eigenvector at the connected vertices are oosite. It is generally the case that the grah Lalacian eigenvectors associated with larger eigenvalues contain more zero crossings, yielding a notion of frequency to the grah Lalacian eigenvalues. However, desite this broader notion of frequency, the grah Lalacian eigenvectors are not always globally-suorted, ure oscillations like the comlex exonentials. In articular, they can feature shar eaks, meaning that some of the Fourier basis elements can be much more similar to an element of the canonical basis of Kronecker deltas on the vertices of the grah. As we will see, uncertainty rinciles for signals on grahs are highly affected by this henomenon. One way to comare a grah Fourier basis to the canonical basis is to comute the coherence between these two reresentations. Definition (Grah Fourier Coherence µ G ). Let G be a grah of N vertices. Let {δ i } i {,2,...,N} denote the canonical basis of l 2 (C N ) of Kronecker deltas and let {u l } l {0,,...,N } be the orthonormal basis of eigenvectors of the grah Lalacian of G. The grah Fourier coherence is defined as: µ G = max i,l δ i, u l = max i,l u l (i) = max s (u l ). l This quantity measures the similarity between the two sets of vectors. If the sets ossess a common vector, then µ G = (the maximum ossible value for µ G ). If the two sets are maximally incoherent, such as the canonical and Fourier bases in the standard discrete setting, then µ G = / N (the minimum ossible value). Because the grah Lalacian matrix encodes the weights of the edges of the grah, the coherence µ G clearly deends on the structure of the underlying grah. It remains an oen question exactly how structural roerties of weighted grahs such as the regularity, clustering, modularity, and other sectral roerties can be linked to the concentration of the grah Lalacian eigenvectors. For certain classes of random grahs [50, 5, 52] or large regular grahs [53], the eigenvectors have been shown to be non-localized, globally oscillating functions (i.e., µ G is low). Yet, emirical studies such as [32] show that grah Lalacian eigenvectors can be highly concentrated (i.e., µ G can be close to ), articularly when the degree of a vertex is much higher or lower than the degrees of other vertices in the grah. The following examle illustrates how µ G can be influenced by the grah structure. Examle. In this examle, we discuss two classes of grahs that can have grah Fourier coherences. The first, called comet grahs, are studied in [33, 54]. They are comosed of a star with k vertices connected to a center vertex, and a single branch of length greater than one extending from one neighbor of the center vertex (see Figure 3, to). If we fix the length of the longer branch (it has length 0 in Figure 3), and increase k, the number of neighbors of the center vertex, the grah Lalacian eigenvector associated with the largest eigenvalue aroaches a Kronecker delta centered at the center vertex of the star. As a consequence, the coherence between the grah Fourier and the canonical bases aroaches as k increases. The second class are the modified ath grahs, which we use several times in this contribution. We start with a standard ath grah of 0 nodes equally saced (all edge weights are equal to one) and we move the first node out to the left; i.e., we reduce the weight between the first two nodes (see Figure 3, bottom). The weight is related to the distance by W 2 = /d(, 2) with d(, 2) being the distance between nodes and 2. When the weight between nodes and 2 decreases, the eigenvector associated with the largest eigenvalue of the Lalacian becomes more concentrated, which increases the coherence µ G. These two examles of simle families of grahs illustrate that the toology of the grah can imact the grah Fourier coherence, and, in turn, uncertainty rinciles that deend on the coherence. In Figure 4, we dislay the eigenvector associated with the largest grah Lalacian eigenvalue for a modified ath grah of 00 nodes, for several values of the weight W 2. Observe that the shae of the eigenvector has a shar local change at node. Examle demonstrates an imortant oint to kee in mind. A small local change in the grah structure can greatly affect the behavior of one eigenvector, and, in turn, a global quantity such as µ G. However, intuitively, a small local change in the grah should not drastically change the rocessing of signal values far away, for examle in a denoising or inainting task. For this reason, in Section 6, we introduce a notion of local uncertainty that deicts how the grah is behaving locally. Note that not only secial classes of grahs or athological grahs yield highly localized grah Lalacian eigenvectors. Rather, grahs arising in alications such as sensor or transortation networks, or grahs constructed from samled manifolds (such as the grah samled from manifold 2 in Figure ) can also have grah Fourier coherences close to (see, e.g., [23, Section 3.2] for further examles). 7

8 Figure 3: Coherence between the grah Fourier basis and the canonical basis for the grahs described in Examle. To left: Comet grahs with k = 6 and k = 2 branches, all of length one excet for one of length ten. To right: Evolution of the grah Fourier coherence µ G with resect to k. Bottom left: Examle of a modified ath grah with 0 nodes. Bottom right: Evolution of the coherence of the modified ath grah with resect to the distance between nodes and 2. As the degree of the comet s N center vertex increases or the first node of the modified ath is ulled away, the coherence µ G tends to the limit value N. Figure 4: Eigenvectors associated with the largest grah Lalacian eigenvalue of the modified ath grah with 00 nodes, for different values of W 2. As the distance between the first two nodes increases, the eigenvector becomes sharly eaked. 3 Global uncertainty rinciles relating the concentration of grah signals in two domains In this section, we derive basic uncertainty rinciles using concentration measures and highlight the limitations of those uncertainty rinciles. 8

9 3. Direct alications of uncertainty rinciles for discrete signals We start by alying three known uncertainty rinciles for discrete signals to the grah setting. Theorem. Let f C N be a nonzero signal defined on a connected, weighted, undirected grah G, let {u l } l=0,,...,n be a grah Fourier basis for G, and let µ G = max i,l δ i, u l. We have the following four uncertainty rinciles: (i) the suort uncertainty rincile [27] (ii) the l -norm uncertainty rincile [30] (iii) the entroic uncertainty rincile [42] (iv) the local uncertainty rincile [34] f 0 + ˆf 0 2 f 0 ˆf 0 µ G, (4) f ˆf µ 2 G f 2 2, [, 2], (5) for any subset V S of the vertices V in the grah G. H(f) + H( ˆf) 2 ln µ G. (6) i V S f(i) 2 V S f 2 V S µ 2 G ˆf 2 (7) The first uncertainty rincile is given by a direct alication of the Elad-Bruckstein inequality [27]. It states that the sarsity of a function in one reresentation limits the sarsity in a second reresentation. As dislayed in (), the work of [27] holds for reresentations in any two bases. As we have seen, if we focus on the canonical basis {δ i } i=,...,n and the grah Fourier basis {u l } l=0,...,n, the coherence µ G deends on the grah toology. For the ring grah, µ G = N, and we recover the result from the standard discrete case (regular samling, eriodic boundary conditions). However, for grahs where µ G is closer to, the uncertainty rincile (4) is much weaker and therefore less informative. For examle, ˆf 0 f 0 is trivially µ 2 G true of nonzero signals. The same caveat alies to (5) and (6), which follow directly from [30] and [42], resectively, by once again secifying the canonical and grah Fourier bases. The last inequality (7) is an adatation [34, Eq. (4.)] to the grah setting, using the Hausdorff-Young inequality of Theorem 2 (see next section). It states that the energy of a function in a subset of the domain is bounded from above by the size of the selected subset and the sarsity of the function in the Fourier domain. If the subset V S is small and the function is sarse in the grah Fourier domain, this uncertainty rincile limits the amount of energy of f that fits insides of the subset of V S. Because V S can be chosen to be a local region of the domain (the grah vertex domain in our case), Folland and Sitaram [34] refer to such rinciles as local uncertainty inequalities. However, the term µ G in the uncertainty bound is not local in the sense that it deends on the whole grah structure and not just on the toology of the subgrah containing vertices in V S. The following examle illustrates the relation between the grah, the concentration of a secific grah signal, and one of the uncertainty rinciles from Theorem. We return to this examle in Section 3.3 to discuss further the limitations of these uncertainty rinciles featuring µ G. Examle 2. Figure 5 shows the comutation of the quantities involved in (5), with = and different G s taken to be the modified ath grahs of Examle, with different distances between the first two vertices. We show the lefthand side of (5) for two different Kronecker deltas, one centered at vertex, and one centered at vertex 0. We have seen in Figure 3 that as the distance between the first two vertices increases, the coherence increases, and therefore the lower bound on the right-hand side of (5) decreases. For δ, the uncertainty quantity on the left-hand side of (5) follows a similar attern. The intuition behind this is that as the weight between the first two vertices decreases, a few of the eigenvectors start to have local jums around the first vertex (see Figure 4). As a result, we can sarsely reresent δ as a linear combination of those eigenvectors and δ is reduced. However, since there are not any eigenvectors that are localized around the last vertex in the ath grah, we cannot find a sarse linear combination of the grah Lalacian eigenvectors to reresent δ 0. Therefore, its uncertainty quantity on the left-hand side of (5) does not follow the behavior of the lower bound. 9

10 Figure 5: Numerical illustration of the l -norm uncertainty rincile on a sequence of modified ath grahs with different mutual coherences between the canonical basis of deltas and the grah Lalacian eigenvectors. For each modified ath grah, the weight W 2 of the edge between the first two vertices is the recirocal of the distance shown on the horizontal axis. The black crosses show the lower bound on the right-hand side of (5), with =. The blue and red lines show the corresonding uncertainty quantity on the left-hand side of (5), for the grah signals δ and δ 0, resectively. 3.2 The Hausdorff-Young inequalities for signals on grahs The classical Hausdorff-Young inequality [43, Section IX.4] is a fundamental harmonic analysis result behind the intuition that a high degree of concentration of a signal in one domain (time or frequency) imlies a low degree of concentration in the other domain. This relation is used in the roofs of the entroy and l -norm uncertainty rinciles in the continuous setting. In this section, as we continue to exlore the role of µ G and the differences between the Euclidean and grah settings, we extend the Hausdorff-Young inequality to grah signals. Theorem 2. Let µ G be the coherence between the grah Fourier and canonical bases of a grah G. Let, q > 0 be such that + q =. For any signal f CN defined on G and 2, we have Conversely, for 2, we have ˆf q µ 2 q G f. (8) ˆf q µ 2 q G f. (9) The roof of Theorem 2, given in Sec. 8., is an extension of the classical roof using the Riesz-Thorin interolation theorem. In the classical (infinite dimensional) setting, the inequality only deends on and q. On a grah, it deends on µ G and hence on the structure of the grah. Dividing both sides of each inequality in Theorem 2 by f 2 leads to bounds on the concentrations (or sarsity levels) of a grah signal and its grah Fourier transform. Corollary. Let, q > 0 be such that + q =. For any signal f CN defined on the grah G, we have s (f)s q ( ˆf) µ 2 q G. Theorem 2 and Corollary assert that concentration or sarsity level of a grah signal in one domain (vertex or grah sectral) limits the concentration or sarsity level in the other domain. However, once again, if the coherence µ G is close to, the result is not articularly informative as s (f)s q ( ˆf) is trivially uer bounded by. The following numerical exeriment illustrates the quantities involved in the Hausdorff-Young inequalities for grah signals. We again see that as the grah Fourier coherence increases, signals may be simultaneously concentrated in both the vertex domain and the grah sectral domain. 0

11 Examle 3. Continuing with the modified ath grahs of Examles and 2, we illustrate the bounds of the Hausdorff-Young inequalities for grah signals in Figure 6. For this examle, we take the signal f to be δ, a Kronecker delta centered on the first node of the modified ath grah. As a consequence, δ = for all, which makes it easier to comare the quantities involved in the inequalities. For this examle, the bounds of Theorem 2 are fairly close to the actual values of ˆδ q. (a) (b) Figure 6: Illustration of the bounds of the Hausdorff-Young inequalities for grah signals on the modified ath grahs with f = δ. (a) The quantities in (8) and (9) for q =, 4 3, 4, and. (b) The quantities in Corollary for the same values of q. Sharness of the grah Hausdorff-Young inequalities. For = q = 2, (8) and (9) becomes equalities. Moreover, for = or =, there is always at least one signal for which the inequalities (8) and (9) become equalities, resectively. Let i and l satisfy µ G = max i,l u l (i) = u l (i ). For =, let f = δ i. Then f =, and ˆf = max l δ i, u l = µ G, and thus (8) is tight. For =, let f = u l. Then f = µ G, ˆf = û l =, and thus (9) is tight. The red curve and its bound in Figure 6 show the tight case for = and q =. 3.3 Limitations of global concentration-based uncertainty rinciles in the grah setting The motivation for this section was twofold. First, we wanted to derive the uncertainty rinciles for grah signals analogous to some of those that are so fundamental for signal rocessing on Euclidean domains. However, we also want to highlight the limitations of this aroach (the second family of uncertainty rinciles described in Section ) in the grah setting. The grah Fourier coherence is a global arameter that deends on the toology of the entire grah. Hence, it may be greatly influenced by a small localized changes in the grah structure. For examle, in the modified ath grah examles above, a change in a single edge weight leads to an increased coherence, and in turn significantly weakens the uncertainty rinciles characterizing the concentrations of the grah signal in the vertex and sectral domains. Such examles call into question the ability of such global uncertainty rinciles for grah signals to accurately describe henomena in inhomogeneous grahs. This is the rimary motivation for our investigation into local uncertainty rinciles in Section 6. However, before getting there, we consider global uncertainty rinciles from the third family of uncertainty rinciles described in Section that bound the concentration of the analysis coefficients of a grah signal in a time-frequency transform domain. 4 Grah signal rocessing oerators and dictionaries As mentioned in Section, uncertainty rinciles can inform dictionary design. In the next section, we resent uncertainty rinciles characterizing the concentration of the analysis coefficients of grah signals in different transform domains. We focus on three different classes of dictionaries for grah signal analysis: (i) frames, (ii) localized sectral grah filter frames, and (iii) grah Gabor filter bank frames, where localized sectral grah filter frames are a subclass of frames, and grah Gabor filter bank

12 frames are a subclass of localized sectral grah filter frames. In this section, we define these different classes of dictionaries, and highlight some of their mathematical roerties. Note that our notation uses dictionary atoms that are double indexed by i and k, but these could be combined into a single index j for the most general case. Definition 2 (Frame). A dictionary D = {g i,k } is a frame if there exist constants A and B called the lower and uer frame bounds such that for all f C N : A f 2 2 f, g i,k 2 B f 2 2. i,k If A = B, the frame is said to be a tight frame. For more roerties of frames, see, e.g., [55, 56, 57]. Most of the recently roosed dictionaries for grah signals are either orthogonal bases (e.g., [6, 5, 20]), which are a subset of tight frames, or overcomlete frames (e.g., [3, 23, 22]). In order to define localized sectral grah filter frames, we need to first recall one way to generalize the translation oerator to the grah setting. Definition 3 (Generalized localization/translation oerator on grahs [3, 23]). We localize (or translate) a kernel ĝ to center vertex i {, 2,..., N} by alying the localization oerator T i, whose action is defined as T i g(n) = N N l=0 ĝ(λ l )u l (i)u l (n). Note that this generalized localization oerator is a kernelized oerator. It does not translate an arbitrary signal defined in the vertex domain to different regions of the grah, but rather localizes a attern defined in the grah sectral domain to be centered at different regions of the grah. The smoothness of the kernel ĝ( ) to be localized can be used to bound the localization of the translated kernel around a center vertex i; i.e., if a smooth kernel ĝ( ) is localized to center vertex i, then the magnitude of T i g(n) decays as the distance between i and n increases [3, Section 5.2], [23, Section 4.4]. Excet for secial cases such as when G is a circulant grah with µ G = N and the Lalacian eigenvectors are the DFT basis, the generalized localization oerator of Definition 3 is not isometric. Rather, we have Lemma ([23], Lemma ). For any g C N, which yields the following uer bound on the oerator norm of T i : where ν i = max l u l (i). ĝ(0) T i g 2 Nν i ĝ 2 Nµ G ĝ 2, (0) T i o = su g C N T i g 2 ĝ 2 Nν i Nµ G, It is interesting to note that although the norm is not reserved when a kernel is localized on an arbitrary grah, it is reserved on average when translated to searately to every vertex on the grah: N N N T i g 2 2 = i= N i= l=0 ĝ(λ l )ū l (i) 2 = N l=0 ĝ(λ l ) 2 N i= ū l (i) 2 = ĝ 2 2. () The following examle resents more recise insights on the interlay between the localization oerator, the grah structure, and the concentration of localized functions. Examle 4. Figure 7 illustrates the effect of the grah structure on the norms of localized functions. We take the kernel to be localized to be a heat kernel of the form ĝ(λ l ) = e τλ l, for some constant τ > 0. We localize the kernel ĝ to be centered at each vertex i of the grah with the oerator T i, and we comute and lot their l 2 -norms T i g 2. The figure shows that when 2

13 λ` Figure 7: The heat kernel g (λ` ) = e 0 λmax (uer left), and the norms of the localized heat kernels, { Ti g 2 }i=,2,...,n, on various grahs. For each grah and each center node i, the color of vertex i is roortional to the value of kti gk2. Within each grah, nodes i that are relatively less connected to their neighborhood seem to yield a larger norm kti gk2. a center node i and its surrounding vertices are relatively weakly connected, the `2 -norm of the localized heat kernel is large, and when the nodes are relatively well connected, the norm is smaller. Therefore, the norm of the localized heat kernel may be seen as a measure of vertex centrality. Moreover, in the case of the heat kernel, we can relate the `2 -norm of Ti g to its concentration s (T g). Localized heat kernels are comrised entirely of nonnegative comonents; i.e., Ti g(n) 0 for all i and n. This roerty comes from (i) the fact that Ti g(n) = (g (L))in (see [3]), and (ii) the non-trivial roerty that the entries of g (L) are always nonnegative for the heat kernel [58]. Since Ti g(n) 0 for all i and n, we have kti gk = N X Ti g(n) = N g (0) = N, (2) n= where the second equality follows from [23, Corollary ]. Thus, recalling that a large value for s (Ti g) means that Ti g is concentrated, we can combine (0) and (2) to derive an uer bound on the concentration of Ti g: s (Ti g) = Ti g 2 Ti g 2 = νi g 2. Ti g N Thus, Ti g 2 serves as a measure of concentration, and according to the numerical exeriments of Figure 7, localized heat kernels centered on the relatively well-connected regions of a grah tend to be less concentrated than the ones centered on relatively less well-connected areas. Intuitively, the values of the localized heat kernels can be linked to the diffusion of a unit of energy from the center vertex to surrounding vertices over a fixed time. In the well-connected regions of the grah, energy diffuses faster, making the localized heat kernels less concentrated. The main class of dictionaries for grah signals that we consider is localized sectral grah filter frames. In fact, the square norm of the localized heat kernel at vertex i is, u to constants, the average diffusion distance from i to all other vertices. It is therefore a genuine measure of centrality. 3

14 Definition 4 (Localized sectral grah filter frame). Let g = {ĝ 0 ( ), ĝ ( ),..., ĝ K ( )} be a sequence of kernels (or filters), where each ĝ k : σ(l) C is a function defined on the grah Lalacian sectrum σ(l) of a grah G. Define the quantity G(λ) := K k=0 ĝ k(λ l ) 2. Then D g = {g i,k } = {T i g k } is a localized sectral grah filter dictionary, and it forms a frame if G(λ) > 0 for all λ σ(l). In ractice, each filter ĝ k ( ) is often defined as a continuous function over the interval [0, λ max ] and then alied to the discrete set of eigenvalues in σ(l). The following lemma characterizes the frame bounds for a localized sectral grah filter frame. Lemma 2 ([22], Lemma ). Let D g = {g i,k } = {T i g k } be a localized sectral grah filter frame of atoms on a grah G generated from the sequence of filters g = {ĝ 0 ( ), ĝ ( ),..., ĝ K ( )}. The lower and uer frame bounds for D g are given by A = N min λ σ(l) G(λ) and B = N max λ σ(l) G(λ), resectively. If G(λ) is constant over σ(l), then D g is a tight frame. Examles of localized sectral grah filter frames include the sectral grah wavelets of [3], the Meyer-like tight grah wavelet frames of [59, 6], the sectrum-adated wavelets and vertex-frequency frames of [22], and the learned arametric dictionaries of [60]. The dictionaries constructions in [3, 22] choose the filters so that their energies are localized in different sectral bands. Different choices of filters lead to different tilings of the vertex-frequency sace, and can for examle lead to wavelet-like frames or vertex-frequency frames (analogous to classical windowed Fourier frames). The frame condition that G(λ) > 0 for all λ σ(l) ensures that these filters cover the entire sectrum, so that no band of information is lost during analysis and reconstruction. In this aer, in order to generalize classical windowed Fourier frames, we often use a localized grah sectral filter bank where the kernels are uniform translates, which we refer to as a grah Gabor filter bank. Definition 5 (Grah Gabor filter bank). When the K kernels used to generate the localized grah sectral filter frame are uniform translates of each other, we refer to the resulting dictionary as a grah Gabor filter bank or a grah Gabor filter frame. If we use the waring technique of [22] on these uniform translates, we refer to the resulting dictionary as a sectrum-adated grah Gabor filter frame. Grah Gabor filter banks are generalizations of the short time Fourier transform. When ĝ is smooth, the atoms are localized in the vertex domain. In this contribution, for all grah Gabor filter frames, we use the following mother window: ĝ(t) = sin ( 0.5π cos(π(t 0.5)) 2), for t [ 0.5, 0.5] and 0 elsewhere. A few desirable roerties of this choice of window are (a) it is erfectly localized in the sectral domain in [ 0.5, 0.5], (b) it is smooth enough to be aroximated by a low order olynomial, and (c) the frame formed by uniform translates (with an even overla) is tight. Definition 6 (Analysis oerator). The analysis oerator of a dictionary D = {g i,k } to a signal f C N is given by A D f(i, k) = f, g i,k. When D = {g i,k } = {T i g k } is a localized sectral grah filter frame, we denote it with A g. In all cases, we view A D as a function from C N to C D, and thus we use A D f (or A g f ) to denote a vector norm of the analysis coefficients. 5 Global uncertainty rinciles bounding the concentration of the analysis coefficients of a grah signal in a transform domain 5. Discrete version of Lieb s uncertainty rincile Lieb s uncertainty rincile in the continuous one-dimensional setting [45] states that the cross-ambiguity function of a signal cannot be too concentrated in the time-frequency lane. In the following, we transose these statements to the discrete eriodic setting, and then generalize them to frames and signals on grahs. The following discrete version of Lieb s uncertainty rincile is artially resented in [6, Proosition 2]. 4

15 Theorem 3. Define the discrete Fourier transform (DFT) as ˆf[k] = N ( ) i2πkn f[n] ex, N N n=0 and the discrete windowed Fourier transform (or discrete cross-ambiguity function) as (see, e.g., [35, Section 4.2.3]) A DDW F T f[u, k] = N n=0 For two discrete signals of eriod N, we have for 2 < and for 2 A DDW F T f = A DDW F T f = ( N N u= k=0 ( N N u= k=0 ( i2πkn f[n]g[n u] ex N ). A DDW F T f[u, k] ) N f 2 g 2, (3) A DDW F T f[u, k] ) N f 2 g 2. (4) These inequalities are roven in Section of the Aendix. Note that the minimizers of this uncertainty rincile are the so-called "icket fence" signals, trains of regularly saced diracs. 5.2 Generalization of Lieb s uncertainty rincile to frames Theorem 4. Let D = {g i,k } be a frame of atoms in C N, with lower and uer frame bounds A and B, resectively. For any signal f C N and any 2, we have A D f B For any signal f C N and any 2, we have A D f A Combining (5) and (6), for any [, ], we have s (A D f) Bmin{ 2, } A max{ 2, } When D is a tight frame with frame bound A, (7) reduces to ( ) 2 max g i,k 2 f 2. (5) i,k ( ) 2 max g i,k 2 f 2. (6) i,k s (A D f) A ( ) 2 2 max g i,k 2. i,k ( ) 2 max g i,k 2. (7) i,k A roof is included in Section 8.2. of the Aendix. The roof of Theorem 3 in Section of the Aendix also demonstrates that this uncertainty rincile is indeed a generalization of the discrete eriodic variant of Lieb s uncertainty rincile. 5

16 5.3 Lieb s uncertainty rincile for localized sectral grah filter frames Lemma imlies that max i,k T i g k 2 Nµ G max k ĝ k 2. Therefore the following is a corollary to Theorem 4 for the case of localized sectral grah filter frames. Theorem 5. Let D g = {g i,k } = {T i g k } be a localized sectral grah filter frame of atoms on a grah G generated from the sequence of filters g = {ĝ 0 ( ), ĝ ( ),..., ĝ K ( )}. For any signal f C N on G and for any [, ], we have s (A g f) Bmin{ 2, } A max{ 2, } ( ) 2 max g i,k 2 Bmin{ 2, } ( ) NµG 2 max ĝ i,k A max{ 2, } k 2, (8) k where A = min λ σ(l) G(λ) is the lower frame bound and B = max λ σ(l) G(λ) is the uer frame bound. When D is a tight frame with frame bound A, (8) reduces to s (A g f) A ( ) 2 2 max g i,k 2 i,k A ( ) NµG 2 2 max ĝ k 2. (9) k The bounds deend on the frame bounds A and B, which are fixed with the design of the filter bank. However, in the tight frame case, we can choose the filters in a manner such that the bound A does not deend on the grah structure. For examle, if the ĝ k are defined continuously on the interval [0, λ max ] and M k=0 ĝ k(λ) 2 is equal to a constant for all λ, A is not affected by a change in the values of the Lalacian eigenvalues, e.g., from a change in the grah structure. The second quantity, max i,k g i,k 2, reveals the influence of the grah. The maximum l 2 -norm of the atoms deends on the filter design, but also, as discussed reviously in Section 4, on the grah toology. However, the bound is not local as it deends on the maximum g i,k 2 over all localizations i and filters k, which takes into account the entire grah structure. The second bounds in (8) and (9) also suggest how the filters can be designed so as to imrove the uncertainty bound. The quantity ĝ k 2 = ( l ĝ k(λ l ) 2) deends on the distribution of the eigenvalues λ l, and, as consequence, on the grah structure. However, the distribution of the eigenvalues can be taken into account when designing the filters in order to reduce or cancel this deendency [22]. In the following examle, we comute the first uncertainty bound in (9) for different tyes of grahs and filters. It rovides some insight on the influence of the grah toology and filter bank design on the uncertainty bound. Examle 5. We use the techniques of [22] to construct four tight localized sectral grah filter frames for each of eight different grahs. Figure 8 shows an examles of the four sets of filters for a 64 node sensor network. For each grah, two of the sets of filters (b and d in Figure 8) are adated via waring to the distribution of the grah Lalacian eigenvalues so that each filter contains an aroriate number of eigenvalues (roughly equal in the case of translates and roughly logarithmic in the case of wavelets). The waring avoids filters containing zero or very few eigenvalues at which the filter has a nonzero value. These tight frames are designed such that A = N, and thus Theorem 5 yields s (A g f) = A gf A g f 2 N 2 max i,k T ig k 2 µ G max ĝ k 2. k Table dislays the values of the first concentration bound max i,k T i g k 2 for each grah and frame air. The uncertainty bound is largest when the grah is far from a regular lattice (ring or ath). As exected, the worst cases are for highly inhomogeneous grahs like the comet grah or a modified ath grah with one isolated vertex. The choice of the filter bank may also decrease or increase the bound, deending on the grah. The uncertainty rincile in Theorem 5 bounds the concentration of the grah Gabor transform coefficients. In the next examle, we examine these coefficients for a series of signals with different vertex and sectral domain localization roerties. Examle 6 (Concentration of the grah Gabor coefficients for signals with varying vertex and sectral domain concentrations.). In Figure 9, we analyze a series of signals on a random sensor network of 00 vertices. Each signal is created by localizing a kernel ĥτ (λ) = e λ 2 λ 2 τ 2 max to be centered at vertex (circled in black). To generate the four different signals, we vary the value 6

17 Grah Gabor Filter Frame Sectrum-Adated Grah Gabor Filter Frame Log-Wared Tight Grah Wavelet Frame Sectrum-Adated Tight Grah Wavelet Frame λ (a) λ (b) λ (c) λ (d) Figure 8: Four different filter bank designs of [22], shown for a random sensor network with 64 nodes. Each colored curve is a filter defined continuously on [0, λ max ], and each filter bank has 6 such filters. They are designed such that G(λ) = for all λ (black line), and thus all four designs yield tight localized sectral grah filter frames. The frame bounds here are A = B = N. Grah Gabor Sectrum-Adated Log-Wared Sectrum-Adated Grah µ G (Uniform Translates) Grah Gabor Wavelets Wavelets Ring Random sensor network Random regular Erdos Renyi Comet Path Modified ath: W 2 = Modified ath: W 2 = Table : Numerical values of the uncertainty bound max i,k T i g k 2 of Examle 5 for various grahs of 64 nodes. of the arameter τ in the heat kernel. We lot the four localized kernels in the grah sectral and vertex domains in the first two columns, resectively. The more we comress ĥ in the grah sectral domain (i.e. we reduce its sectral sreading by increasing τ), the less concentrated the localized atom becomes in the vertex domain. The joint vertex-frequency reresentation A g T h τ (i, k) of each signal is shown in the third column, which illustrates the trade-off between concentration in the vertex and the sectral domains. The concentration of these grah Gabor transform coefficients is the quantity bounded by the uncertainty rincile resented in Theorem 5. In the last row of the Figure 9, τ = which leads to a Kronecker delta for the kernel and a constant on the vertex domain. On the contrary, when the kernel is constant, with τ = 0 (to row), the energy of the grah Gabor coefficients stays concentrated around one vertex but sreads along all frequencies. 6 Local uncertainty rinciles for signals on grahs In the revious section, we defined a global bound for the concentration of the localized sectral grah filter frame analysis coefficients. In the classical setting, such a global bound is also local in the sense that each art of the domain has the same structure, due to the regularity of the underlying domain. However, this is not the case for the grah setting where the domain is irregular. Examle shows that a bad structure (a weakly connected node) in a small region of the grah reduces the uncertainty bound even if the rest of the grah is well behaved. Functions localized near the weakly connected node can be highly concentrated in both the vertex and frequency domains, whereas functions localized away from it are barely imacted. Imortantly, the worst case determines the global uncertainty bound. As another examle, suose one has two grahs G and 7

18 τ = 0 τ = 4 τ = 0 τ = Figure 9: Grah Gabor transform of four different signals f τ = T h τ, with each row corresonding to a signal with a different value of the arameter τ. Each of the signals is a kernel localized to vertex, with the kernel to be localized equal to ĥτ (λ) = e λ 2 λ 2 max τ 2. The underlying grah is a random sensor network of 00 vertices. First column: the kernel h τ (λ) is shown in red and the localized kernel f τ is shown in blue, both in the grah sectral domain. Second column: the signal f τ in the vertex domain (the center vertex is circled). Third column: A g T h τ (i, k), the absolute value of the Gabor transform coefficients for each vertex i and each of the 20 frequency bands k. Fourth column: since it is hard to see where on the grah the transform coefficients are concentrated when the nodes are laced on a line in the third column, we dislay the value 9 k=0 A gt h τ (i, k) on each vertex i in the network. This figure illustrates the tradeoff between the vertex and the frequency concentration. G 2 with two different structures, each of them having a different uncertainty bound. The uncertainty bound for the grah G that is the union of these two disconnected grahs is the minimum of the uncertainty bounds of the two disconnected grahs, which is subotimal for one of the two grahs. In this section, we ask the following questions. Where does this worse case haen? Can we find a local rincile that more accurately characterizes the uncertainty in other arts of the grah? In order to answer this question, we investigate the concentration of the analysis coefficients of the frame atoms, which are localized signals in the vertex domain. This technique is 8

19 used in the classical continuous case by Lieb [45], who defines the (cross-) ambiguity function, the STFT of a short-time Fourier atom. The result is a joint time-frequency uncertainty rincile that does not deend on the localization in time or in frequency of the analyzed atom. Thus, we start by generalizing to the grah setting the definition of ambiguity (or cross-ambiguity) functions from timefrequency analysis of one-dimensional signals. Definition 7 (Ambiguity function). The ambiguity function of a localized sectral frame D = {g i,k } = {T i g k } is defined as: A g (i 0, k 0, i, k) = A g T i0 g k0 (i, k) = T i0 g k0, T i g k When the kernels {ĝ k } k=0,,...,m are aroriately wared uniform translates, the oerator A g becomes a generalization of the short time Fourier transform. Additionally, the ambiguity function assesses the degree of coherence (linearly deendence) between the atoms T i0 g k0 and T i g k. In the following, we use this ambiguity function to robe locally the structure of the grah, and derive local uncertainty rinciles. 6. Local uncertainty rincile In order to robe the local uncertainty of a grah, we take a set of localized kernels in the grah sectral domain and center them at different local regions of the grah in the vertex domain. The atoms resulting from this construction are jointly localized in both the vertex and grah sectral domains, where "localized" means that the values of the function are zero or close to zero away from some reference oint. By ensuring that the atoms are localized or have suort within a small region of the grah, we focus on the roerties of the grah in that region. In order to get a local uncertainty rincile, we aly the frame oerator to these localized atoms, and analyze the concentration of the resulting coefficients. In doing so, we develo an uncertainty rincile relating these concentrations to the local grah structure. To reare for the theorem, we first state a lemma that gives a hint to how the scalar roduct of two localized functions deends on the grah structure and roerties. In the following, we multily two kernels ĝ and ĥ in the grah sectral domain. For notation, we reresent the roduct of these two kernels in vertex domain as g h. Lemma 3. For two kernels ĝ, ĥ and two nodes i, j, the localization oerator satisfies and ( i < T i g, T j h > = NT i (g h)(j) (20) < T i g, T j h > ) = N T j (g h). (2) Equation (20) shows more clearly the conditions on the kernels and nodes under which the scalar roduct is small. Let us take two examles. First, suose ĝ and ĥ have a comact suort on the sectrum and do not overla (kernels localized in different laces), then ĝ ĥ is zero everywhere on the sectrum, and therefore the scalar roduct on the left-hand side of (20) is also equal to zero. Second, assume i and j are distant from each other. Then T i (g h)(j) is small if ĝ and ĥ) are reasonably smooth. In other words, the two atoms T i g and T j h must be localized both in the same area of grah in the vertex domain and the same sectral region in order for the scalar roduct to be large. This localization deends on the atoms, but also on the grah structure. Proof of Lemma 3. < T i g, T j h > = < T i g, T j h >= N = N N l=0 N l=0 ĝ(λ l )u l (i) ĥ(λ l )ū l (j) ( ) ĝ ĥ (λ l )u l (i)ū l (j) = NT i (g h)(j). 9

20 Moreover, a direct comutation shows ( i < T i g, T j h > ) = ( ) NT j (g h)(i) i = N 2 Tj (g h). The following theorem rovides inequalities giving a local uncertainty rincile. The local bound deends of the localization of the atom T i0 g k0 both in the grah and sectral domains. The center vertex i 0 and kernel ĝ k0 can be chosen to be any vertex and kernel; however, the locality roerty of the uncertainty rincile aears when T i0 g k0 is concentrated around node i 0 in the vertex domain and around a small ortion of the sectrum in the grah sectral domain. Once again, we measure the concentration with l -norms. Theorem 6 (Local uncertainty). Let {T i g} {i [,N],k [0,M ]} be a localized sectral grah filter frame with lower frame bound A and uer frame bound B. For any i 0 [, N], k 0 [0, M ] such that T i0 g k0 2 > 0, the quantity A g T i0 g k0 = ( M N k= i= ) < T i g k, T i0 g k0 > = ( M N T i0 (g k0 g k ) k= ) (22) satisfies for [, ] s (A g T i0 g k0 ) Bmin{, } Tĩi0 2,k 0 g ki0,k 0 2 A 2 ( ) B min{, 2 } Nνĩi0,k 0 g ki0,k 2 0 where ν i is defined in Lemma, k Ti0 i0,k 0 = arg max k T i0 (g k0 g k ), and ĩ i0,k 0 = arg max i (g k0 )(i). g ki0,k 0 A 2, (23) The bound in (23) is local, because we get a different bound for each i 0, k 0 air. For each such air, the bound deends on the quantities ĩ i0,k 0, k i0,k 0, which are maximizers over a set of all vertices and kernels, resectively; however, as we discuss in Examle 7 below, ĩ i0,k 0 is tyically close to i 0, and k i0,k 0 is tyically close to k 0. For this reason, this bound tyically deends only on local quantities. Proof of Theorem 6. For notational brevity in this roof, we omit the indices i 0, k 0 for the quantities ĩ and k. First, note that A g T i0 g k0 = max N Ti0 (g k0 g k ) k Tĩg k 2 T i0 g k0 2, where k i0,k 0 = arg max k T i0 (g k0 g k ) and ĩ i0,k 0 = arg min i T i0 (g k0 g k)(i). Let us then interolate the two following exressions: A g T i0 g k0 2 B 2 Ti0 g k0 2 (24) and A g T i0 g k0 Tĩg k 2 T i0 g k0 2. (25) We use the Riesz-Thorin Theorem (Theorem 8) with = q = 2 = 2, q 2 =, M = B 2 and M q = Tĩg k 2. Note that A g is a bounded oerator from the Hilbert sace sanned by T i0 g k0 (isomorhic to a one-dimensional Hilbert sace) to the one sanned by {T i0 g k0 } i,k. We take t = 2 r 2 and find r = 2, leading to A g T i0 g k0 r2 B r 2 Tĩg k 2 r 2 2 T i0 g k

21 Since A g is a frame, we also have A g T i0 g k0 2 A 2 T i0 g k0 2, which yields: A g T i0 g k0 2 A g T i0 g k0 Finally, thanks to Hölder s inequality, we have for 2 and + q = A 2 B Tĩg k 2 2 A g T i0 g k0 2 A g T i0 g k0 A gt i0 g k0 q A g T i0 g k0 2 B q Tĩg k 2 2 A 2 B Tĩg k 2 2 A 2 B ( Nνĩ g k 2) 2 A 2.. The next corollary shows that in many cases, the local uncertainty inequality (23) is shar (becomes an equality). To obtain this, we require that the frame A g is tight and T i g k, T i0 g k0 is maximized when k = k 0 and i = i 0. Corollary 2. Under the assumtions of Theorem 6 and, assuming additionally. A g is a tight frame with frame-bound A, 2. k 0 = arg max k T i0 (g k g k0 ), and 3. i 0 = arg max j T i0 g 2 k 0 (j), we have s (A g T i0 g k0 ) = A gt i0 g k0 = T i 0 g k0 2. (26) A g T i0 g k0 2 Proof. The roof follows directly from the two following equalities. For the denominators, since the frame is tight, we have: For the numerators, we have A g T i0 g k0 2 = A 2 Ti0 g k0 2. A 2 A g T i0 g k0 = max i,k T ig k, T i0 g k0 = N max i,k T i 0 (g k g k0 )(i) (27) = N max T i0 (g k g k0 ) k = N T i0 g 2 k 0 (28) = N T i0 g 2 k 0 (i 0 ) (29) = T i0 g k0, T i0 g k0 (30) = T i0 g k0 2 2, where (27) and (30) follow from (20), (28) follows from the second hyothesis, and (29) follows from the third hyothesis. 2

22 Corollary 3. Under the assumtions of Theorem 6, we have which is a lower bound on the concentration measure. Proof. We have s (A g T i0 g k0 ) = A gt i0 g k0 T i 0 g k0 2, (3) A g T i0 g k0 2 A g T i0 g k0 = max i,k T ig k, T i0 g k0 T i0 g k0, T i0 g k0 = T i0 g k (32) Additionally, because {T i g k } i=,2,...,n;k=0,,...,m is a frame, we have Combining (32) and (33) yields the desired inequality in (3). B 2 A g T i0 g k0 2 B 2 Ti0 g k0 2. (33) Together, Theorem 6 and Corollary 3 yield lower and uer bounds on the local sarsity levels s (A g T i0 g k0 ): Tĩg k 2 A 2 s (A g T i0 g k0 ) = A gt i0 g k0 T i 0 g k0 2. A g T i0 g k0 2 B Illustrative examles In order to better understand this local uncertainty rincile, we illustrate it with some examles. Examle 7 (Local uncertainty on a sensor network). Let us concentrate on the case where =. Theorem 6 tells us that ( ) Nνĩi0,k0 g ki0,k0 2 A g T i0 g k0 A g T i0 g k0 2 T ĩ i0,k0 g ki0,k 2 0 A 2 A 2, (34) meaning that the concentration of A g T i0 g k0 is limited by Tĩg ki0,k 0 2. One question is to what extent this quantity is local or reflects the local behavior of the grah. As a general illustration for this discussion, we resent in Fig. 0 quantities related to the local uncertainty of a random sensor network of 00 nodes evaluated for two different values of k (one in each column) and all nodes i. 22

23 Grah Gabor filter bank First filter k0 = 0 (blue) Second Filter k0 = (orange) Local sarsity level: s (Ag Ti0 gk0 ) = kag Ti0 gk0 k kag Ti0 gk0 k2 Uer bound on local sarsity from Theorem 6: kti i 0,k0 gk i A 0,k0 k2 2 Maximimizing filter index: k k0,i0 Ho distance between i k0,i0 and i0 Lower bound on the local sarsity from Corollary 3: kti0 gk0 k2 B2 Relative error between s (Ag Ti0 gk0 ) and B 2 kti0 gk0 k2 : s (Ag Ti0 gk0 ) B 2 kti0 gk0 k2 s (Ag Ti0 gk0 ) Figure 0: Illustration of Theorem 6 and related variables i and k for a random sensor grah of 00 nodes. To figure: the 8 uniformly translated kernels {gbk }k (in 8 different colors) defined on the sectrum and giving a tight frame. Each row corresonds to quantities related to the local uncertainty rincile. The first column concerns the kernel (filter) in blue on the to figure, the second is associated with the orange one. On a sensor grah, the local uncertainty level (inversely roortional to the local sarsity level lotted here) is far from constant from one node to another or from one frequency band to another. 23

24 The first row (not counting the to figure) shows the local sarsity levels of A g T i0 g k0 in terms of the l -norm (left hand side of (34)) at each node of the grah. The second row shows the values of the uer bound on local sarsity for each node of the grah (middle term of (34)). The values of both rows are strikingly close. Note that for this tye of grah, local sarsity/concentration is lowest where the nodes are well connected. We focus now on the values of k and ĩ as they are crucial in Theorem 6. We also give insights that exlain when a tight bound is obtained, as stated in Corollary 2. There is not a simle way to determine the value of k, because it deends not only on the node i 0 and the filters ĝ k, but also on the grah Fourier basis. However, the definition k = arg max k T i0 (g k g k0 ) imlies that the two kernels ĝ k and ĝ k0 have to overla as much as ossible in the grah Fourier domain in order to maximize the infinity-norm. In the case of a Gabor filter bank like the one resented in the first line of Fig. 0, k 0 = k for most of the nodes. This haens because the filters ĝ k and ĝ k0 do not overla much if k k 0, i.e when ĝ k = l ( ) 2 ĝ 2 k0 (λ l )» (ĝ k0 (λ l )ĝ k (λ l )) 2 = ĝ k ĝ k l In fact, in the case of Fig. 0, k is bounded between k 0 and k 0 + because there is no overla with the other filters. In Fig. 0, we lot k(i) for k 0 = 0 and k 0 =. For the first filter, we have k i0,k 0 = k 0 for all vertices i 0. The second filter follows the same rule excet for two nodes. The isolated node on the north east is less connected to the rest and there is a Lalacian eigenvector well localized on it. As a consequence, the localization on the grah is affected in a counter-intuitive manner. Let us now concentrate on the second imortant variable: ĩ. Under the assumtion that the kernels ĝ k are smooth, the energy of localized atoms T i0 g k reside inside a ball centered at i 0 [23]. Thus, the node j maximizing T i0 (g g k)(j) k0 cannot be far from the node i 0. Let us define the ho distance h G (i, j) as the length of the shortest ath 2 between nodes i and j. If the kernels ĝ k are olynomial functions of order K, the localization oerator T i0 concentrates all of the energy of T i0 g k inside a K-radius ball centered in i 0. Since the resulting kernel ĝ k0 ĝ k is a olynomial of order 2K, ĩ will be at a distance of at most of 2K hos from the node i 0. In general, ĩ is close to i 0. In fact, the distance h G (i, ĩ) is related to the smoothness of the kernel ĝ k0 ĝ k [23]. To illustrate this effect, we resent in Fig. the average and maximum ho distance h G (i, ĩ). In this examle, we control the concentration of a kernel ĝ with a dilation arameter a: ĝ a (x) = ĝ(ax). Increasing a comresses the kernel in the Fourier domain and increases the sread of the localized atoms in the vertex domain. Note that even for high sectral comression, the ho distance h G (i, ĩ) remains low. Additionally, we also comute the mean relative error between T i g 2 and T i g 2 (i). This quantity asserts how well T i g 2 2 estimates T i g 2. 3 Returning to Fig. 0, the fourth row shows the ho distance between i 0 and ĩ. It never exceed 3 for both the first and the second filter, which is a good sign of locality. In ractice we can not always determine the values of k and ĩ, but as we have seen, the quantity B 2 T i g k0 2 may still be a good estimate of the local sarsity level. Row 5 of Fig. 0 shows these estimates, and the last row shows the relative error between these estimates and the actual local sarsity levels. We observe that for the first kernel, the estimate gives a sufficiently rough aroximation of the local sarsity levels. For the second kernel, the aroximation error is low for most of the nodes, but not all. In the next examle, we comare the local and global uncertainty rinciles on a modified ath grah. Examle 8. On a 64 node modified ath grah (see Examle for details), we comute the grah Gabor transform of the signals f = T g 0 and f 2 = T 64 g 0. In Figure 2, we show the evolution of the grah Gabor transforms of the two signals with resect to the distance d = /W 2 from the first to the second vertex in the grah. As the first node is ulled away, a localized eigenvector aears centered on the isolated vertex. Because of this, as this distance increases, the signal f becomes concentrated in both the vertex and grah sectral domains, leading to grah Gabor transform coefficients that are highly concentrated (see the to right lot in Fig. 2). However, since the grah modification is local, it does not drastically affect the grah Gabor transform coefficients of the signal f 2 (middle row of Fig. 2), whose energy is concentrated on the far end of the ath grah. 2 A ath in a grah is a tule of vertices (v, v 2,..., v ) with the roerty that [v i, v i+ ] E for i. Two nodes v i, v j are connected by a ath if there is exist such tule with v = v i and v = v j. The length of a ath is defined as the cardinality of the ath tule minus one. 3 From Lemma 3, when T i g 2 = T i g 2 (i), then T i g 2 = T i g

25 T a mean ig T ig(i) i T ig in % mean i h G (ĩ, i) max i h G (ĩ, i) Heat kernel Wavelet kernel Figure : Localization exeriment using the sensor grah of Fig. 0. The heat kernel is defined as ĝ(ax) = e 0 ax λmax and the wavelet kernel ĝ(ax) = 40 ax e 40 ax λmax. For a smooth kernel ĝ, the ho distance h G between i and ĩ = arg max j T i g(j) is small. In Figure 3, we lot the evolution of the uncertainty bounds as well as the concentration of the Gabor transform coefficients of f and f 2. The global uncertainty bound from Theorem 5 tells us that The local uncertainty bound from Theorem 6 tells us that s (A g f) max i,k T ig k 2, for any signal f. s (A g T i0 g k0 ) Tĩi0,k 0 g ki0,k 0 2, for all i 0 and k 0. Thus, we can view the global uncertainty bound as an uer bound on all of the local uncertainty bounds. In fact the bums in the global uncertainty bound in Figure 3 corresond to the local bound with i 0 = and different frequency bands k 0. We lot the local bounds for i 0 = and k 0 = 0 and k 0 = Single kernel analysis Let us focus on the case where we analyze a single kernel ĝ. Such an analysis is relevant when we model the signal as a linear combination of different localizations of a single kernel: f(n) = N w i T i g(n) i= This model has been roosed in different contributions [62, 63, 64], and has also been used as an interolation model, e.g., in [65] and [24, Section V.C]. In this case, we could ask the following question. If we measure the signal value at node j, how much 25

26 d = d d 7 d = 27 d = 8 f = T g 0 f 2 = T 64 g 0 f = T g 0 Vertex domain Figure 2: Grah Gabor transforms of f = T g 0 and f 2 = T 64 g 0 for 5 different distances between vertices and 2 of the modified ath grah. The distance d = /W 2 is the inverse of the weight of the edge connecting the first two vertices in the ath. The node 64 is not affected by the change in the grah structure, because its energy is concentrated on the oosite side of the ath grah. The grah Gabor coefficients of f, however, become highly concentrated as a grah Lalacian eigenvector becomes localized on vertex as the distance increases. The bottom row shows that as the distance between the first two vertices increases, the atom T g 0 also converges to a Kronecker delta centered on vertex. Figure 3: Concentration of the grah Gabor coefficients of f = T g 0 and f 2 = T 64 g 0 with resect to the distance between the first two vertices in the modified ath grah, along with the uer bounds on this concentration from Theorem 5 (global uncertainty) and Theorem 6 (local uncertainty). Each bum of the global bound corresonds to a local bound of a given sectral band of node. For clarity, we lot only bands ĝ 0 and ĝ 2 for node. For node 64, the local bound is barely affected by the change in grah structure, and the sarsity levels of the grah Gabor transform coefficients of T 64 g 0 also do not change much. information do we get about w j? We can answer this by looking at the overla between the atom T j g and the other atoms. When T j g has a large overla with the other atoms, the value of f(j) does not tell us much about w j. However, in the case where T j g has a very small overla with the other atoms (an isolated node for examle), knowing f(j) gives an excellent aroximation for 26

27 the value of w j. The following theorem uses the sarsity level of g(l)t j g to analyze the overla between the atom T j g and the other atoms. Theorem 7. For a kernel ĝ, the overla between the atom localized to center vertex j and the other atoms satisfies O (j) = ( i < T ig, T j g > ) ( i < T ig, T j g > 2) 2 = g(l)t jg g(l)t j g 2 = T jg 2 T j g 2 2 Proof. This result follows directly from the alication of (2) in Lemma Alication: non-uniform samling Examle 9 (Non-uniform samling for grah inainting). In order to motivate Theorem 7 from a ractical signal rocessing oint of view, we use it to otimize the samling of a signal over a grah. To asses the quality of the samling, we solve a small inainting roblem where only a art of a signal is measured and the goal is to reconstruct the entire signal. Assuming that the signal varies smoothly in the vertex domain, we can formulate the inverse roblem as: argmin x T Lx s. t. y = Mx, (35) x where y is the observed signal, M the inainting masking oerator and x T Lx the grah Tikhonov regularizer (L being the Lalacian). In order to generate the original signal, we filter Gaussian noise on the grah with a low ass kernel ĥ. The frequency content of the resulting signal will be close to the shae of the filter ĥ. For this examle, we use the low ass kernel ĥ(x) = to generate the smooth signal x λmax For a given number of measurements, the traditional idea is to randomly samle the grah. Under that strategy, the measurements are distributed across the network. Alternatively, we can use our local uncertainty rinciles to create an adated mask. The intuitive idea that nodes with less uncertainty (higher local sarsity values) should be samled with higher robability because their value can be inferred less easily from other nodes. Another way to icture this fact is the following. Imagine that we want to infer a quantity over a random sensor network. In the more densely oulated arts of the network, the measurements are more correlated and redundant. As result, a lower samling rate is necessary. On the contrary, in the arts where there are fewer sensors, the information has less redundancy and a higher samling rate is necessary. The heat kernel ĝ(x) = e τx is a convenient choice to robe the local uncertainty of a grah, because ĝ2 (x) = e 2τx is also a heat kernel, resulting in a sarsity level deending only on T j g 2 2. Indeed we have T j g 2 = N. The local uncertainty bound of Theorem 7 becomes: O (j) = T jg 2 N T j g 2 = 2 T j g 2. 2 Based on this measure, we design a second random samled mask with a robability roortional to T i g 2 2 ; that is, the higher the overla level at vertex j, the smaller the robability that vertex j is chosen as a samling oint, and vice-versa. For each samling ratio, we erformed 00 exeriments and averaged the results. For each exeriment, we also randomly generated new grahs. The exeriment was carried out using oen-source code: the UNLocBoX [66] and the GSPBox [67]. Figure 4 resents the result of this exeriment for a sensor grah and a community grah. In the sensor grah, we observe that our local measure of uncertainty varies smoothly on the grah and is higher in the more dense art. Thus, the likelihood of samling oorly connected vertices is higher than the likelihood of samling well connected vertices. In the community grah, we observe that the uncertainty is highly related to the size of the community. The larger the community, the larger the uncertainty (or, equivalently, the smaller the local sarsity value). In both cases, the adated, non-uniform random samling erforms better than random uniform samling. Other works are also starting to use uncertainty rinciles to develo samling theory for signals on grahs. In [68], the cumulative coherence is used to otimize the samling distribution. This can be seen as samling roortionally to T i g 2 2, 27

28 kti gk2 samling distribution (Sensor network) kti gk2 samling distribution (Community grah) Reconstruction error (Sensor network) Reconstruction error (Community grah) (a) (b) (c) (d) Smooth signal Samle locations (Uniform samling) Samle locations (Non-uniform samling) Reconstruction (Uniform samling) Reconstruction (Non-uniform samling) (e) (f) (g) (h) (i) Figure 4: Comarison of random uniform samling and random non-uniform samling according to a distribution based on the local sarsity values. To row: (a)-(b) The random non-uniform samling distribution is roortional to kti gk2 (for different values of i), which is shown here for sensor and community grahs with 300 vertices. (c)-(d) the errors resulting from using the different samling methods on each grah, with the reconstruction in (35). Bottom row: an examle of a single inainting exeriment. (e) the smooth signal, (f)-(g) the locations selected randomly according to the uniform and non-uniform samling distributions, (h)-(i) the reconstructions resulting from the two different sets of samles. where g is a secific rectangular kernel, in order to minimize the cumulative coherence of band-limited signals. In [40], Tsitsvero et al. make a link between uncertainty and samling to obtain a non-robabilistic samling method. While non-uniform random samling is only an illustrative examle in this aer, we are currently working on a searate contribution that uses our uncertainty theory to otimize samling. 7 Conclusion The global uncertainty rinciles discussed in Section 3 may be less informative when alied to signals residing on inhomogeneous grahs, because the structure of a secific area of the grah can affect global quantities such as the coherence µg, which lay a key role in the uncertainty bounds. Our main contribution was to suggest a new way of considering uncertainty by incororating a notion of locality; secifically, we focused on the concentration of the analysis coefficients under a linear transform whose dictionary atoms are generated by localizing kernels defined in the grah sectral domain to different areas of the grah. The equivalent hysical aroach would be to say that the uncertainty on the measurements deends on the medium where the article is located. Comaring the first inequality in (23) from the local uncertainty Theorem 6 with the first inequality in (8) from the global uncertainty Theorem 5, we see that the latter global bound can be viewed as the maximum of the local bounds over all regions of the grah and all regions of the sectrum.4 This suorts our view that the benefit of the global uncertainty 4 The leading constants in the middle terms of (8) and (23) are equal for > 2. When < 2, there is a constant factor bounds. This factor is equal to in the case of a tight frame (A = B). 28 B A 2 between the two