Witsenhausen s counterexample as Assisted Interference Suppression

Transcription

1 Witsenhausen s counterexaple as Assisted Interference Suppression Pulkit Grover and Anant Sahai Wireless Foundations, epartent of EECS University of California at Berkeley, CA-947, USA {pulkit, sahai}@eecs.berkeley.edu Abstract espite the seeingly irreducible siplicity of Witsenhausen s counterexaple, the optial control law for the proble is not yet known. It has been observed that the counterexaple contains an iplicit counication proble. We forulate a vector version of the Witsenhausen counterexaple that helps siplify the proble further while preserving its essential character. The vector version of the counterexaple is shown to correspond to a previously unstudied counication proble which we dub Assisted Interference Suppression. Using concepts of lossy copression, channel coding, and dirtypaper coding, nonlinear control strategies are developed that outperfor the optial linear laws in the liit of large vector lengths for this new proble. The proposed techniques also outperfor the best affine strategy and the known scalar strategies by an unbounded factor. A new lower bound is also developed using inforation-theoretic arguents that is soeties better than that derived by Witsenhausen. I. INTROUCTION istributed control is hard. Moreover, it is not even clear why exactly it is hard. Thus, the quest for a deeper understanding has naturally focused on trying to understand the precise nature of the boundary that separates easy probles fro hard ones. In this quest, Linear Quadratic Gaussian (LQG) systes have played a special role. This is because perfectly classical inforation patterns, it is well known that controllers affine in the observation are optial for such systes. By way of a counterexaple, Witsenhausen [] deonstrated that the property fails to hold for distributed systes. He designed a inialist two-step LQG distributed control syste with two controllers operating at different tie steps. For this syste, he provided a nonlinear control law that perfors better than the optial affine control laws. He also showed that a easurable optial control law ust exist. The literature here related to the Witsenhausen counterexaple is reasonably rich. We liit our discussion to the few papers that are ost relevant to our discussion and in setting a perspective on the proble. In [], a discrete version of Witsenhausen s proble is introduced. This allows for a convex forulation over a set of coplicated constraints, and thus suggests that the proble ay be easy to solve. However, in [3], the discrete version was shown to be NP coplete. Soe approaches for searching for the optial solution are Part of this work has been subitted to Conference on ecision and Control (CC) 8. A part of this work has been subitted to 47 th IEEE Conference on ecision and Control, Cancun, 8, Mexico. obtained in (aongst other works) [4] [6] using tools fro inforation theory, neural networks and stochastic optiization respectively. Since the search proble is nonconvex, this body of work has also inspired nuerical ethods for solving nonconvex probles. Nevertheless, the optial solution is still elusive. A different approach to understanding the boundary between hard and easy probles is adopted in another body of work that attepts to show optiality of affine control for certain distributed LQG systes. In [4], for exaple, the authors consider a paraetrized faily of two-stage stochastic control probles. The faily includes the Witsenhausen counterexaple. Using results fro inforation theory, the authors show that for this faily, whenever the cost function does not contain a product of two decision variables, affine control laws are still optial. In [7], the author shows that affine control laws continue to be optial for a deterinistic variant of the Witsenhausen counterexaple if the cost function is the induced two-nor instead of the expected two-nor in the stochastic variant. We instead ask the following question: Is the proble posed by Witsenhausen really the siplest counterexaple? At first, it ight see that the siplest counterexaple is the one with the fewest nuber of variables. Witsenhausen hiself states There does not appear to exist any counterexaple involving fewer variables... than the one presented in []. Contrary to this intuition, in this paper, we argue that the vector version of the proble has the potential to actually siplify the proble by allowing us to sidestep the geoetry of finitediensional spaces by allowing an asyptotic law-of-largenubers perspective by taking the liit of large vector lengths. The precise proble itself is posed in Section II. Our starting point is the observation that the nonlinear control law in [] akes use of counication fro the first controller to the second controller. This fact was ade explicit in [8]. The first controller observes the initial state noiselessly and can ake changes to the state. The new state is then observed noisily by the second controller. The key observation is that by aking appropriate changes to the initial state, controller can counicate to controller via this noisy state channel. In [8], using this crucial insight, the authors designed schees based on quantization of the initial state. When the second controller observes a noisy version of this state, the quantization helps in eliinating the noise in its observation. By varying the choice of the proble paraeters,

2 the authors provide a sequence of probles for which certain quantization-based strategies outperfor the optial linear strategies by a factor that tends to infinity. This counication analogy is taken ore seriously in Section III where it is shown that the vector Witsenhausen counterexaple is just an inforation theoretic proble in disguise. This proble is dubbed Assisted interference suppression. Although the proble itself has not been studied earlier, it is shown to be one of a natural faily of probles of which others have been studied quite extensively in the inforation-theoretic literature. Section IV then proposes a couple of nonlinear schees building on the scalar quantization ideas introduced in [8]. More precisely, the quantization in [8] is perfored by a strategy that is conceptually related to Tolinson-Harashia precoding (see e.g. [9, Pg.454]). Tolinson-Harashia precoding is a scalar version for what is called dirty-paper coding in inforation theory. Ideas of dirty-paper coding are used here for the vector proble. A lower bound is also developed based on inforation theoretic ideas since Witsenhausen s original lower bound of [, Section 6] is tied to the scalar proble. There is a gap between our lower bounds and our upper bounds, thus indicating that ore work is needed. However, if indeed the gap can be closed, then we would have a coplete understanding of the vector version of Witsenhausen s proble. This would capture ost of the distributed-control tensions that are contained by the proble. Understanding the original Witsenhausen counterxaple would then reduce to understanding the difference between scalar and vector versions of the proble. That sae sort of gap in understanding exists in ost inforation-theoretic probles, but is arguably less interesting. II. THE VECTOR VERSION OF WITSENHAUSEN S COUNTEREXAMPLE We generalize the scalar Witsenhausen proble to a vector case. The syste is still a two-step control syste. The states and the inputs are now vectors of length. A vector is represented in bold font, with the superscript used to denote a vector length (e.g. x ). As in conventional notation, x is used to denote states, u the input, and y the observation. The state x is distributed N(, σ I). The state transition functions : x = f (x,u ) = x u, and x = f (x,u ) = x u. The output equations: where w N(, σ wi). The cost expressions: y = g (x ) = x, and y = g (x ) = x w, h (x,u ) = k u, and h (x,u ) = x. The cost expressions are noralized by the vector-length so that they do not grow with the proble size. They are denoted by cost and cost, respectively. The control objective is to iniize the average cost, averaged over the realizations of x and w. The inforation patterns : Y = {y }; U =, Y = {y }; U =. The quantity E[ u ] will often be referred to as the power P of the input. P is proportional to the cost at tie. The perforance is usually evaluated by finding iniu cost for given k. We note that finding the optial perforance for all k is equivalent to finding the optial tradeoff between P and cost, x. To see this, we first observe that the region of achievable (P, cost) pairs is convex. This is because for any two strategies A and B that attain two points (P a, cost a ) and (P b, cost b ), any of their convex cobination (λp a ( λ)p b, λcost a ( λ)cost b ) can be achieved using a randoized strategy that chooses strategy A with probability λ and strategy B with probability λ. Now, the intercept on the cost axis of the tangent of slope k on the P cost tradeoff gives the iniu cost for the given value of k. On the other hand, given iniu attainable average total costs for all k, whether a point lies in P cost achievable region can be found be checking if k P cost is larger than the iniu attainable cost for all k. In this paper, we use either of these representations depending on the which presents the results in a ore lucid anner. III. CONNECTIONS WITH INFORMATION THEORY The proble described in Section II can in fact be viewed as an inforation theory proble. This correspondence is explained through Fig.. The proble illustrated in Fig. (a) is the vector version for Witsenhausen s counterexaple drawn in traditional for with the state evolution foring the backbone of the figure. This is transfored by redrawing the blocks so that the iplicit counication channel fors the backbone of the figure and then suppressing the final state and viewing the final control as a reconstruction of the input to the channel. After all, the controller should use u = ˆx in order to iniize the ean-square error. The resulting proble illustrated in Fig. (b) turns out to be an unstudied proble in inforation theory. The encoder has non-causal knowledge of the interference, and has an average power constraint of P. The objective is to iniize the distortion d(x,ˆx ), where ˆx is the iniu ean-square estiate of x at the decoder. The twist that distinguishes the proble here fro previously considered inforation-theoretic probles is that the syste is allowed to ake changes to the state x and the distortion is calculated between the new state x and the reconstruction of x. However, it is possible to interpret this atheatical proble in the wireless counication scenario. Fig. illustrates the proble, which is why we refer to it as Assisted Interference Suppression. The transitter T has to

3 3 counicate its essage to receiver R in presence of a huge interference fro I. The interferer generates an iid Gaussian signal x that is known non-causally at the helper H. The helper attepts to suppress the interference at the receiver R. The signal received at R is given by y = x u w z where w is the signal transitted by T, and z is the AWGN. In order to suppress the effect of x, the helper H could transit x. However, if P < σ, this is not possible. Thus better techniques can be used to decode or estiate x u. An inforation theoretic proble is to characterize the region of achievable rates for the transitterreceiver syste. A good strategy for the helper is to choose the input u within the power constraint such that it helps provide a good estiate of x = x u. This is exactly the objective of the vector Witsenhausen proble. The proble is, however, soewhat unrealistic, because it assues that the helper H knows the channel fade coefficient fro the interferer to the receiver. (See [], [] for why this provides a challenge in a parallel context soeties called the cognitive channel by inforation theorists.) However, it shows that the vector Witsenhausen proble should naturally be a proble of interest to inforation theorists as well. () Interferer I x Helper H x u Receiver R w' Transitter T Fig.. The figure shows the Assisted Interference Suppression proble, a odel for which is given in Fig. 3. The special case of R = is an inforation theoretic counterpart of Witsenhausen s proble. The helper H has non-causal knowledge of the interference x. It attepts to suppress the interference at receiver R (by odifying it using u ) so that the receiver R can get a clean channel fro the transitter T. z x x u C C (a) w - u x M rate R H E u w' x x = x u z ^ M E u x x E [ u ] = P (b) w ^x Fig.. In (a), the vector version of the Witsenhausen counterexaple is posed as a control proble. The total cost is given by k u x, and the objective is to iniize the average cost. In (b), the sae proble is cast as an inforation theory proble. The input to the channel u has an average power constraint of P, that is, E[ u ] P. The encoder E has noncausal side inforation of the interference x. The objective is to iniize the average distortion E[d(x,ˆx )]. The two probles are equivalent. At this point, we do not have a proof of the optiality of this strategy. However, for the particular case when the MMSE error can be ade zero, it is evident that this strategy is optial. Fig. 3. The figure shows inforation theoretic odel for the Assisted Interference Suppression proble shown in Fig.. The objective is to axiize the rate R. For the helper, a strategy can be polish the dirt so that the the decoder can obtain a good estiate estiate x, thus cleaning the channel for use by the encoder E. Closely related probles have been addressed in the inforation theory literature. All these probles are based on Fig. (b). We now discuss three such related probles. The irty-paper Coding (PC) proble [] addresses the proble of counicating a essage reliably across a channel with known interference. The interference vector is assued known non-causally at the transitter. The objective is to counicate the essage reliably at axiu possible rate R for a given transit power P. A tradeoff between R and P is calculated. Surprisingly, it turns out that this tradeoff is the sae as that for a channel with no interference. In this context, the Assisted Interference Suppression proble can also be interpreted as one coponent in a distributed dirty-paper coding proble. One agent, the helper, has non-

4 4 causal knowledge of the interference. A different agent, the transitter, wants a clean channel for its signal. The helper and the transitter ust ipleent dirty-paper coding in a distributed anner. The helper polishes the dirt, so that it can be ore easily suppressed by the receiver that sees the su of the polished dirt, receiver noise, and the desired essage. Recently, a different related proble of state aplification was solved by Ki et al [3]. For the syste in Fig. (b), the objective in [3] is to convey the inforation of the initial state x of the syste to the receiver, along with a essage. There is an average power constraint on the input, that is, u P. The authors characterize the tradeoff between the rate R for the essage and the obtained reduction in uncertainty in the knowledge of the state at the receiver. The iniization of uncertainty in knowledge of state is observed to be equivalent to iniization of ean-square error for the Gaussian state. For R =, therefore, the objective is to iniize the eansquare error in estiation of x at the receiver. Another variation of the proble that is in soe sense the dual of the proble in [3] is that of state asking [4]. The objective there is to iniize the inforation about x that can be obtained fro y, while siultaneously transitting a essage across the channel. The authors obtain the optial tradeoff between the attainable rates R and the utual inforation I(x,y ). The three proble probles addressed in inforation theory along with the Assisted Interference Suppression proble are all suarized in Table I. The table suggests that these probles are natural cousins of each other. IV. TWO SCHEMES BASE ON INFORMATION-THEORETIC IEAS In this section we design two schees for the vector Witsenhausen proble. The schees are based on inforation theoretic concepts and they are shown to perfor better (in soe cases) than any affine schee, and known scalar schees. For ease of elucidation, we provide results as powercost tradeoffs rather than iniu cost. Finally, to copare various bounds and the perforance of the scalar schee in [8], we plot the total cost curves (for varying proble paraeters). A. A Joint Source-Channel Coding (JSCC) based schee We first provide a nonlinear coding schee that is based on the concept of joint source-channel coding in inforation theory. To enable understanding of the schee, we review soe fundaental results and definitions fro inforation theory in Appendix I. These are taken fro [5], and the reader is referred to [5] for further details. We first briefly describe the schee, before giving a detailed analysis of its perforance. As in [8], the idea is to quantize the space of realizations of x to arrive at x. These points are chosen carefully so that with high probability, the second controller can recover x fro the noisy observation y. By aking u = x, the second controller can now force x, and hence the second cost, to zero. In the vector case, for a careful choice of points, the probability of error in recovering x converges to zero exponentially in [6]. Therefore, for large enough, the average cost at tie can be ade as sall as desired. At tie, the state of the syste is x = x u. We use the following construction to find u for each x. First, we design a rate-r source code for distortion where σ > > σ w. A rando codebook is constructed, with each codeword drawn randoly fro distribution N(, σ I) for σ = σ. If the code rate R satisfies R R() = ( ) σ log, () the average distortion is no greater than (in the liit). We denote the quantization point for given initial state x by ˆx. The choice of u is the distortion ˆx x, and the resulting x = ˆx. Thus x, which is the quantized x, is itself transitted across the channel. Since a rando Gaussian codebook achieves the channel capacity [5] for an average power constraint equal to the average power of the codebook, the points in the codebook for a good channel code as well. Since these codewords are generated N(, σ I), the average power of the codebook is σ = σ. Therefore, x can be recovered reliably at the second controller for rates R < C where C = log ( ) σ σw = ( log σ ) σw. (3) Siplifying the capacity expression, C = ( log σ ) σw = ( σ log w σ ) σ w Thus, for reliable counication, C ( ) σ log = R(), which is satisfied when ( σ log w σ ) log σ w i.e. σ w σ σ w i.e. (σ σ w ) σ w σ ( ) σ, (4) (5) σ. (6) (6) is satisfied iff lies between σ and σw. Thus any > in{σ, σ w } can be attained for this syste. Since is the ean-square distortion E[ x x ], it is also the nor square of the input required to drive x to x. Therefore, the cost at tie is k. Observe that is only constrained by the inequality > in{σ, σ w}. Asyptotically, therefore, the first stage cost is k in{σ, σw}. Since the error probability converges to zero exponentially in, for large enough, the average cost at second stage can If σw σ, then choice of u = x forces the state x to zero. This is equivalent to lossy copression with rate and lies within the fraework of JSCC described here.

5 5 TABLE I A COMPARISON OF THE SET-UPS FOR VARIOUS INFORMATION THEORETIC PROBLEMS. Inforation theoretic proble Input cost constraint Counicate/obfuscate Perforance easures irty Paper Coding [] P Message R State Aplification [3] P Message, x R and d(x,ˆx ) State Masking [4] P Message, hide x R, while iniizing I(x,y ) Assisted Interference Suppression P Message R (by iniizing d(x,ˆx )) be ade as close to zero as desired. Therefore, the asyptotic total cost is just k in{σ, σ w }. The JSCC based schee is conceptually easier to understand using the recently proposed deterinistic odel for counication in wireless networks [7]. In the wireless odel in Fig., the helper H clears out the channel for the second controller by zero forcing the low order bits of the interference, thereby creating space for the transitter T to counicate its essage to the receiver R. In the distributed PC interpretation of the proble, if each of the helper and the essage encoder have equal power, as long as the power exceeds σw, the encoder can counicate reliably. Thus there is at ost a half-bit loss in rate for the transitter-receiver counication when the PC is ipleented in a distributed anner as copared to the ipleentation in the usual joint anner. Note that the above schee has no flexibility the cost factor k is ignored copletely in the design. This otivates design of schees that have a soother tradeoff between the two costs. Also, observe that the lossy source code perfors a quantization on x. This quantization reduces the power3 that is fed into the channel and iposes a constraint of > in{σ, σ w } for reliable counication. The constraint lower bounds k, the cost at tie. It leads to a natural question whether there exist quantization schees that increase the power of the resulting codeword. In the next section, we provide one such schee. B. A irty-paper Coding (PC) based schee In this section, we propose a faily schees based on dirtypaper coding []. This faily is paraeterized by constant α. For a given choice of k, we can optiize over α in order to obtain a good schee. As in the last section, dirty-paper coding (PC) techniques [] in inforation theory can be thought of as perforing a quantization. Contrary to the quantization in the JSCC schee, PC increases the power in the codeword. This suggests that dirty-paper schees ight perfor better than the JSCC schee. We refer the reader to Costa s original paper [] for ore details. We note that due to standard control theory notation used here, our notation is different fro that in []. The schee proceeds by choosing an auxiliary rando variable V N(, P α σ ), for soe α that will be chosen by an optiization later. M = T iid sequences are drawn uniforly at rando fro the set of typical v, where 4 [, 3 This power is E[ x ]. 4 T corresponds to the utual inforation between V and Y []. Eqn. (3)] T = log ( (P σ σw )(P α σ ) ) Pσ ( α) σw (P σ ). (7) These sequences are then distributed uniforly over nr bins. A particular bin is chosen 5. The encoding is now perfored as follows. Given a source sequence x, a v jointly typical with x is first found in the chosen bin. Then u = v αx is chosen as the input at tie. The received sequence y is, therefore y = u x w. (8) It is shown in [] that the decoder (in our case the second controller), can recover v fro the received sequence as long as the rate R is saller than [, Eqn. 6] C(α, P) = log ( P(P σ σ w) Pσ ( α) σ w (P α σ ) ). (9) Since there is no essage to be counicated, as a first step, we can assue the rate R to be zero. This keeps P sall, which lowers the costs since k P is the cost at tie. At tie, the cost is the average ean-square error in estiating x = u x. The decoder can reliably recover v = x αu. Also, decoder observes the channel output y. Therefore, the decoder can estiate u x fro the two observations ζ = u αx, ζ = u x w () By design, u and x are statistically independent 6, and their entries are chosen independently. Therefore, we can find the error in estiating u x fro v by linear MMSE estiation. The total cost is, therefore, k P MMSE. () This cost can be achieved only if C(α, P) in (9) is greater than. Thus, the optial cost is obtained by iniizing () under the constraint that C(α, P) >. Consider α =. In this case, the receiver reliably recovers v = u x = x. By choosing u = x, the cost at tie is zero. Also, C(, P) = ( P(P σ log σ ) w) σw(p σ ), () 5 Eventually we will let R, so there s no loss in choosing any particular bin. 6 The schee [] is designed such that each eleent of u and x appear to have been drawn independently fro their respective Gaussian iid distributions. That is, asyptotically in block length, the epirical correlation between u and x is zero.

6 6 which is strictly positive at P = σw. Therefore, it is possible to ake the second cost zero for soe values of P < σw for this schee. Since the MMSE cost is zero, the net cost is saller than σw. Notice that this was not possible for the JSCC based schee, where the cost is constrained to be greater than σw. We note here that there is potential advantage in counicating at non-zero rate. The first controller can quantize the initial state x using a vector quantizer. It can then send counicate this quantized state reliably across the channel. The receiver thus gets an estiate of x, in addition to the observations in (), and can iprove the ean square estiate of x. However, our epirical observations suggest that the gain here is alost negligible for sall as well as large values of σ. In this work, therefore, we restrict ourselves to zero-rate ipleentation of PC based schee. C. A cobination of the linear schee and the PC based schee While the PC schee outperfors the linear schee for high values of σ, Fig. 4 shows that this is not true for low values of σ. A natural strategy to overcoe this proble is to design a cobination of the linear schee and the PC based schee. A linear ter u, = ax constitutes a part of u. To this, we add a vector u, that is dirty-paper coded against ( a)x. The state x is, therefore, x = ( a)x u,, (3) where u, is the input that dirty-paper codes against ( a)x, and a σ E[ u, ] P. By choosing a σ = P, the schee reduces to the linear schee. On the other hand, choosing a = reduces the schee to a pure PC based schee. Thus the perforance of this cobined schee can be no worse than that of the linear schee or the PC based schee. For σ =.75, Fig. 4 shows that in soe cases the perforance of the cobination schee can indeed be better than the PC based schee as well as the linear schee. The figure suggests that for low power a pure linear schee perfors well, as opposed to the high power case, when a pure PC based approach perfors well. Fig. 5 akes this explicit. For low power, linear perfors better than the PC, and all the power is dedicated to the PC schee. As the power increases, the part dedicated to the PC increases and at large power, all the power is dedicated to the PC schee, and a =.. A lower bound on the costs for the vector proble Witsenhausen [, Section 6] derived a lower bound on the costs for the scalar proble. However, his lower bound does not hold for the vector case. Thus a new lower bound is needed for the vector case. The following theore derives one such lower bound. Theore (Lower bound to the vector proble): Given an average power of P at tie-step, the average distortion E[d(x,ˆx )] (the cost at tie ) is lower bounded Power σ =.75 JSCC Optial linear schee PC linear PC lower bound noralized cost Fig. 4. For sall values of σ, a cobination of the linear schee and the PC based schee can perfor better than both of these schees. For large power, it is possible to force the vector x to zero, and hence the linear schee perfors better. For sall power, PC coding perfors better. JSCC based schee requires a power of.75 for all values of cost. Observe that PC based schee attains zero cost for P <.75. We noralize cost by the axiu possible cost that is attained when P = since in the liit of P, all schees have the sae perforance. η = Fraction of power dedicated to linear part σ = Power Fig. 5. η denotes the fraction of power that is dedicated to the linear schee. For low P, η is close to, iplying that all the power is dedicated to the linear part. As η increases, the power is shared between PC and linear parts, until at high power, when a pure-pc based approach perfors as well as the cobination schee. by E[d(x,ˆx )] { ( κ P ) if κ > P otherwise, (4) where σ κ = σw σ P Pσ σw Proof: See Appendix III E. Coparison with linear and scalar schees (5) In this section, we copare the vector schee with the optial linear schee, and the scalar nonlinear schees in [8].

7 7 Power.5.5 σ =.5 Linear JSCC PC linear PC lower bound Thus any choice of sequence (k, σ ) such that k and σ, the ratio diverges to infinity. Observe that there is ore flexibility in choice of (k, σ ) as copared to that in [8], where a careful choice has been ade. The four schees, viz. the optial linear schee and the three vector nonlinear schees proposed here are copared in Fig. 8 and Fig. 9. In Appendix II, we show that the proposed schee can outperfor the scalar nonlinear schee in [8] by a factor of infinity. This is also evident fro Fig noralized cost Linear JSCC PC Lower bound on schee in [9] Fig. 6. For large σ, PC based schee perfors better than the linear schee, and the optial cobination of linear and PC schee reduces to the PC schee. log (cost).4.35 σ = JSSC n Power PC Lower Bound PC linear linear Fig. 8. This figure shows how the of cost (on a log-log scale) varies with n, where n is the paraeter that characterizes the faily of control probles in [8]. Thus, k n = n, σ,n =.n, and for the schee in [8], the size of bin B n = n. A lower bound on cost for this schee is derived in Appendix II. Since slopes for PC and JSCC costs are better than that for a lower bound on schee in [8], the ratio of costs for the schee in [8] and these schees diverges to infinity noralized cost Fig. 7. For sall σ, PC based schee perfors worse than the linear schee, and the optial cobination of linear and PC schee reduces to the linear schee. For siplicity, assue σ w =. For given value of σ, the cost for the optial linear schee is (fro [8]) inf a k a σ ( a) σ ( a) σ. (6) Since σw =, the asyptotic cost for the JSCC based schee is k σw = k. The ratio of the optial linear cost to the cost for the JSCC based is, therefore, k a σ inf (a) σ (a) σ a k = inf a a σ ( a) k ( a) σ Now let k and σ. If a is close to, the second ter is unbounded. If a is close to, the first ter gets unbounded. For any other value of a, both ters are unbounded. V. ISCUSSIONS AN CONCLUSIONS In this paper we consider the vector version of Witsenhausen s counterexaple. Making use of the iplicit channel in the set-up of the counterexaple, and assuing large vector lengths, we apply nonlinear vector schees inspired by inforation theoretic techniques. These schees are shown to outperfor the optial affine schee and the known scalar schees by a factor that converges to infinity for a sequence of probles. The results reaffir the notion that counication is central to Witsenhausen s counterexaple. The counterexaple is erely an instance, and it suggests that inforation theory ight offer useful tools for obtaining good schees for general distributed control systes, and bounds on their gap fro optiality. We believe that the derived lower bound to the vector proble is loose because it allows for perfect alignent of the input u with the initial state x. Such an alignent would increase the variance of x, and hence also the distortion between the new state x and its reconstruction. However, it decreases the end-to-end distortion d(x, ˆx ). Thus this bound does not capture the tension between aligning u with x and decreasing d(x, ˆx ). An iproveent in the lower bound should thus be possible.

8 8 log (cost) PC 3.5 Witsenhausen s scalar lower bound Vector lower Bound n Fig. 9. Plot of cost as a function of n, with k n = n, σ,n =.n on a log-log scale coparing the lower bounds with the upper bound obtained fro the PC schee. Witsenhausen s scalar lower bound plotted here is not valid for the vector proble. Interestingly, in soe cases, the scalar lower bound falls above the vector upper bound, showing that strictly better perforance can be achieved in the vector case. The figure shows that the vector lower bound derived here is tighter than Witsenhausen s scalar lower bound in certain cases. Since it is also valid for the scalar case, this gives a new lower bound to Witsenhausen s proble. The two schees proposed here provide upper bounds to the proble. The observation that the linear schee perfors better than the PC based schee for low σ suggests that the upper bounds can also be iproved upon. The inforation theoretic proble of Assisted Interference Suppression posed in Section III sees siilar to the other probles discussed in that section. This provides hope that the upper bounds and the lower bounds can be tightened and the exact tradeoff obtained for large vector lengths. We would obtain a solution to the vector version of Witesenhausen s counterexaple in the liit of long vector lengths. This would reduce the proble to that of understanding the effects of finite vector lengths, a proble that is still under investigation in inforation theory. APPENIX I SOME USEFUL INFORMATION THEORETIC CONCEPTS A. Lossy source coding Assue that we have a source that produces sequence x X. The encoder describes the source sequence x by an index f (x ) {,,..., nr }. The decoder represents x by an estiate ˆx ˆX. efinition : A distortion function or distortion easure is a apping d : X ˆX R (7) fro the set of source alphabet-reproduction alphabet pairs into the set of non-negative real nubers. The distortion d(x, ˆx) is a easure of the cost of representing the sybol x by the sybol ˆx. efinition : The distortion between sequences x and ˆx is defined by d(x,ˆx ) = d(x i, ˆx i ) (8) n i= efinition 3: A ( R, ) rate distortion code consists of an encoding function, f : X {,,..., R } (9) and a decoding (reproduction) function, g : {,,..., R } ˆX. () The distortion associated with the ( R, ) code is defined as = E[d(x, g (f (x )))] () where the expectation is with respect to the probability distribution on x. efinition 4: A rate distortion pair (R, ) is said to be achievable if there exists a sequence of ( R, ) rate distortion codes (f, g ) with li E[d(x, g (f (x )))]. The rate-distortion function R() is the infiu of rates R such that (R, ) is achievable for a given distortion. The distortion-rate function (R) is the infiu of all distortions such that (R, ) achievable for a given rate R. Theore (R() for Gaussian source): The ratedistortion function for Gaussian source N(, σ ) with squared-error distortion is { ( ) R() = log σ, σ, > σ. () The proof of this theore tells us that this codebook can be constructed by choosing nr points independently fro N(, (σ )I) distribution. B. Channel coding efinition 5: An Additive White Gaussian Noise (AWGN) channel with an average power constraint consists of a channel input X R and a channel output Y = X Z, where Z N(, σw). The input X has an average power constraint P, that is, over channel uses, i= E [ X i ] P efinition 6: An (M, ) code for the AWGN channel consists of the following: ) An index set {,,..., M}. ) An encoding function X : {,,..., M} R, yielding codewords X (),X (),...,X (M). The set of codewords is called the codebook. 3) A decoding function g : R {,,..., M}, which is a deterinistic rule which assigns a guess to each possible received vector. efinition 7 (Probability of error): Let λ i = Pr(g(Y ) i X = X (i)) (3) be the conditional probability of error given that i was sent. The average probability of error is defined as Pe = M λ i, (4) M i=

9 9 and the axial probability of error is defined as λ () = ax λ i (5) i {,,...,M} efinition 8: A rate R is said to be achievable if there exists a sequence of ( R, ) codes such that the axial probability of error λ () as n. efinition 9: The capacity of a eoryless channel is the supreu of all achievable rates. Theore 3 (Channel coding theore): The capacity for an additive white Gaussian noise channel of noise variance σw with an average power constraint P is C = ( log P ) σw (6) In addition, the error probability converges to zero exponentially in [6], and the capacity can be achieved by a choosing a codebook of R points independently fro N(, PI) distribution. APPENIX II PERFORMANCE COMPARISON WITH SCALAR SCHEME IN [8] For the faily of probles and the quantization schee in [8], we find lower bounds on the cost at tie. We follow the notation of [8] in this section. B is used to denote the -th bin (bin that includes the origin), and B is the bin-size. cost k = E [ (γ B (x )) ] E [ x ] {B } = πσ B/ B/ x e x /σ dx. For the particular sequence of proble paraeters n in [8], the size of n th bin is B n = n, σn = n and k n = n. Therefore, cost n k n = = πn 4 πn 4 n/ n/ n/ n/ x e x /n 4 dx x e n /8n 4 dx n/ x dx e /8n πn 4 n π e /8n, that increases to infinity as n. In coparison, the joint source-channel coding based schee proposed in Section IV has cost of kn. Thus the ratio costn k = for the joint sourcechannel schee. Hence, the ratio of the costs for the scalar n schee in [8] and the vector schee proposed here diverges to infinity. APPENIX III ERIVATION OF THE LOWER BOUN ON THE COST FOR VECTOR WITSENHAUSEN PROBLEM In this section, we derive a lower bound on the cost for the vector Witsenhausen proble. Since the bound is valid for any vector length, it is also valid for =. However, this bound turns out to be looser than the lower bound presented by Witsenhausen in [, Section 6] for =. Since techniques in [] do not generalize to >, this bound is needed to estiate the gap in the perforance for the proposed schees fro the optial. The bound is derived as follows. The end-to-end distortion in the initial state is given by d(x,ˆx ). Suppose we intend to iniize this distortion. This distortion is always lower bounded by the distortion that can be achieved across the channel fro controller to controller. Observe that the channel is AWGN of noise variance σw. The initial state x is Gaussian with noise covariance atrix σi. To attain the iniu distortion across this channel, the initial state should be scaled to the average power constraint of the channel and transitted without any coding. If u has average power P, then the average power across this channel is bounded by σ P σ P, which is attained by aligning u along x. Thus the end-to-end distortion in x across this channel is lower bounded by σ σ w E[d(x,ˆx )] σ P = κ, (7) Pσ σw which is obtained by MMSE estiation of x fro the observation y. Now let A, B and C be any three rando variables. Using the triangle inequality on Euclidian distance, d(b, C) d(a, C) d(a, B), (8) and d(b, C) d(a, B) d(a, C). (9) Thus, d(b, C) d(a, C) d(a, B), (3) Squaring both sides, d(b, C) d(a, C)d(A, B) d(a, C) d(a, B). (3) Taking expectation on both sides, E[d(B, C)] E[d(A, C)] E[d(A, B)] E[ d(a, C) d(a, B)]. (3) Now, using Cauchy-Schwartz inequality, [ ] E d(a, C) d(a, B) E[d(A, C)] E[d(A, B)]. (33) Using (3) and (33), E[d(B, C)] E[d(A, C)] E[d(A, B)] = E[d(A, C)] E[d(A, B)] ( ) E[d(A, C)] E[d(A, B)]. Substituting x for A, x for B, and ˆx for C, we get ) E[d(x,ˆx )] (E[d(x,ˆx )] E[d(x,x )]. (34) If κ > P, using E[d(x, ˆx )] κ and E[d(x,x )] P, E[d(x,ˆx )] ( κ P ). (35) If κ P, then we lower bound E[d(x,ˆx )] by zero.

10 REFERENCES [] H. S. Witsenhausen, A counterexaple in stochastic optiu control, SIAM Journal on Control, vol. 6, no., pp. 3 47, Jan [] Y.-C. Ho and T. Chang, Another look at the nonclassical inforation structure proble, IEEE Transactions on Autoatic Control, 98. [3] C. H. Papadiitriou and J. N. Tsitsiklis, Intractable probles in control theory, SIAM Journal on Control and Optiization, vol. 4, no. 4, pp , 986. [4] R. Bansal and T. Basar, Stochastic teas with nonclassical inforation revisited: When is an affine control optial? IEEE Transactions on Autoatic Control, 987. [5] M. Baglietto, T. Parisini, and R. Zoppoli, Nonlinear approxiations for the solution of tea optial control probles, Proceedings of the IEEE Conference on ecision and Control CC, pp , 997. [6] J. T. Lee, E. Lau, and Y.-C. L. Ho, The witsenhausen counterexaple: A hierarchical search approach for nonconvex optiization probles, IEEE Transaction on Autoatic Control, vol. 46, no. 3,. [7] M. Rotkowitz, Linear controllers are uniforly optial for the witsenhausen counterexaple, Proceedings of the 45th IEEE Conference on ecision and Control CC, pp , ec. 6. [8] S. K. Mitter and A. Sahai, Inforation and control: Witsenhausen revisited, in Learning, Control and Hybrid Systes: Lecture Notes in Control and Inforation Sciences 4, Y. Yaaoto and S. Hara, Eds. New York, NY: Springer, 999, pp [9]. Tse and P. Viswanath, Fundaentals of Wireless Counication. New York: Cabridge University Press, 5. [] P. Grover and A. Sahai, On the need for knowledge of the phase in exploiting known priary transissions, in IEEE yspan 7, ublin, Ireland, Oct. 7. [], Writing on rayleigh faded dirt: a coputable upper bound to the outage capacity, in Proceedings of the 7 IEEE Syposiu on Inforation Theory, Nice, France, Jul. 7. [] M. Costa, Writing on dirty paper, IEEE Trans. Infor. Theory, vol. 9, no. 3, pp , May 983. [3] Y.-H. Ki, A. Sutivong, and T. M. Cover, State aplification, IEEE Trans. Infor. Theory, vol. 54, no. 5, pp , May 8. [4] N. Merhav and S. Shaai, Inforation rates subject to state asking, IEEE Trans. Infor. Theory, vol. 53, no. 6, pp. 54 6, Jun. 7. [5] T. M. Cover and J. A. Thoas, Eleents of Inforation Theory. New York: Wiley, 99. [6] R. G. Gallager, Inforation Theory and Reliable Counication. New York, NY: John Wiley, 97. [7] A. Avestiehr, S. iggavi, and. Tse, A deterinistic approach to wireless relay networks, in Proc. of the Allerton Conference on Counications, Control and Coputing, October 7.