Communication, Computation, and Control: Through the Information Lens

Transcription

1 Communication, Computation, and Control: Through the Information Lens Pulkit Grover Stanford University CMU, April 16, Acknowledgments: C2S2/Interconnect Focus Center (IFC) NSF Center for Science of Information (CSoI) 1 NSF-CCF (cyber-physical systems)

2 Flows of information in modern systems 2/33

3 Flows of information in modern systems 2/33

4 Flows of information in modern systems 2/33

5 Flows of information in modern systems 2/33

6 Talk outline: Understanding Invisible Information Flows 1) Information flows in circuits M Transmitter P T Channel Receiver M 3/33

7 Talk outline: Understanding Invisible Information Flows 1) Information flows in circuits M Transmitter P T Channel Receiver M 3/33

8 Talk outline: Understanding Invisible Information Flows 1) Information flows in circuits M Transmitter P T Channel Receiver M 2) Information flows in physical systems C w1 C w2 C w6 Plant C w3 C w5 C w4 3/33

9 Talk outline: Understanding Invisible Information Flows 1) Information flows in circuits M Transmitter P T Channel Receiver M 2) Information flows in physical systems C w1 C w2 C w6 Implicit channel Plant C w3 C w5 C w4 3/33

10 Talk outline: Understanding Invisible Information Flows 1) Information flows in circuits M Transmitter P T Channel Receiver M Traditional: Minimize P T Now: Minimize total power Fundamental limits, experimental results 2) Information flows in physical systems C w1 C w2 C w6 Implicit channel Plant C w3 C w5 C w4 3/33

11 Short-distances: Processing power can exceed transmit power! Short-distance communication at Gbps (e.g. indoor wireless, data-centers ethernet) Encoder PA LNA ADC Equalizer Decoder M 80 mw [1] 11 mw[4] 26 mw[2] 12 mw [1] 144 mw [3] Transmitter Receiver [1] Uncoded [Marcu et al '09] [2] 7-bit ADCs [Murmann '08] [3] (6,32)-LDPC [Zhang et al '10] [4] [Shin 09] 4/33

12 Short-distances: Processing power can exceed transmit power! Short-distance communication at Gbps (e.g. indoor wireless, data-centers ethernet) Encoder PA LNA ADC Equalizer Decoder M 80 mw [1] 11 mw[4] 26 mw[2] 12 mw [1] 144 mw [3] Transmitter Choosing a -efficient code? P total Receiver [1] Uncoded [Marcu et al '09] [2] 7-bit ADCs [Murmann '08] [3] (6,32)-LDPC [Zhang et al '10] [4] [Shin 09] 4/33

13 Short-distances: Processing power can exceed transmit power! Short-distance communication at Gbps (e.g. indoor wireless, data-centers ethernet) Encoder PA LNA ADC Equalizer Decoder M 80 mw [1] 11 mw[4] 26 mw[2] 12 mw [1] 144 mw [3] Transmitter Choosing a -efficient code? P total Receiver P total = P T + P enc + P dec [1] Uncoded [Marcu et al '09] [2] 7-bit ADCs [Murmann '08] [3] (6,32)-LDPC [Zhang et al '10] [4] [Shin 09] 4/33

14 Minimizing transmit power: Shannon s waterfall curves Goal: Rate M E P T R, error probability 5 fixed R, W, η P e D M C = W log path loss 1+ ηp T N 0 >R log (P ) 10 (Pe) e Shannon limit (Tx power) Total power (Watts) Transmit Total power (Watts) (watts) 5/33

15 Minimizing total power: Initial theoretical approaches M E P T D M 6/33

16 Minimizing total power: Initial theoretical approaches Black-box model of processing power M E P T D M [Cui, Goldsmith, Bahai 05] [Massaad, Medard, Zheng 08] 6/33

17 Minimizing total power: Initial theoretical approaches Black-box model of processing power M Optimal burstiness E P T D M [Cui, Goldsmith, Bahai 05] [Massaad, Medard, Zheng 08] 6/33

18 Minimizing total power: Initial theoretical approaches Black-box model of processing power M Optimal burstiness E P T D M [Cui, Goldsmith, Bahai 05] [Massaad, Medard, Zheng 08] 0 T 6/33

19 Minimizing total power: Initial theoretical approaches Black-box model of processing power M Optimal burstiness E P T D M [Cui, Goldsmith, Bahai 05] [Massaad, Medard, Zheng 08] 0 T 6/33

20 Minimizing total power: Initial theoretical approaches Black-box model of processing power M Optimal burstiness E P T D M [Cui, Goldsmith, Bahai 05] [Massaad, Medard, Zheng 08] 0 T 6/33

21 Minimizing total power: Initial theoretical approaches Black-box model of processing power M Optimal burstiness E P T D M [Cui, Goldsmith, Bahai 05] [Massaad, Medard, Zheng 08] 0 T What are the greenest codes? 6/33

22 Green system = Capacity-code + Green implementation? Theory Implementation On the design of LDPC codes within db of the Shannon limit. [Chung, Forney, Richardson, Urbanke IT 01] A 47-Gbps LDPC decoder... (10 GBase-T) [Zhang, Anantharam, Wainwright, Nikolic JSSC 10] 7/33

23 Green system = Capacity-code + Green implementation? Theory Implementation On the design of LDPC codes within db of the Shannon limit. [Chung, Forney, Richardson, Urbanke IT 01] A 47-Gbps LDPC decoder... (10 GBase-T) [Zhang, Anantharam, Wainwright, Nikolic JSSC 10] Wireless LAN and 60 GHz band 7/33

24 Green system = Capacity-code + Green implementation? Theory Implementation On the design of LDPC codes within db of the Shannon limit. [Chung, Forney, Richardson, Urbanke IT 01] A 47-Gbps LDPC decoder... (10 GBase-T) [Zhang, Anantharam, Wainwright, Nikolic JSSC 10] Wireless LAN and 60 GHz band uncoded transmission! 7/33

25 What can the total power look like? Some experiments 8/33

26 What can the total power look like? Some experiments Circuit-simulation-based modeling of decoding power [Ganesan, Grover, Rabaey SiPS 11][Ganesan, Wen, Grover, Rabaey 12] 8/33

27 What can the total power look like? Some experiments Circuit-simulation-based modeling of decoding power [Ganesan, Grover, Rabaey SiPS 11][Ganesan, Wen, Grover, Rabaey 12] log 10 (Pe) Shannon waterfall Power (Watts) 8/33

28 What can the total power look like? Some experiments Circuit-simulation-based modeling of decoding power [Ganesan, Grover, Rabaey SiPS 11][Ganesan, Wen, Grover, Rabaey 12] log 10 (Pe) Shannon waterfall Regular LDPCs (3,4) (3,6) (4,8) (5,10) P total Power (Watts) R = 7 Gbps W = 7 GHz path-loss coeff. = 3 d = 3 meters fc = 60 GHz STM CMOS 90 nm Cadence Encounter (Layout) Synopsis HSPICE 8/33

29 What can the total power look like? Some experiments Circuit-simulation-based modeling of decoding power [Ganesan, Grover, Rabaey SiPS 11][Ganesan, Wen, Grover, Rabaey 12] log 10 (Pe) P T Shannon waterfall Regular LDPCs (3,4) (3,6) (4,8) (5,10) P total Power (Watts) R = 7 Gbps W = 7 GHz path-loss coeff. = 3 d = 3 meters fc = 60 GHz STM CMOS 90 nm Cadence Encounter (Layout) Synopsis HSPICE 8/33

30 What can the total power look like? Some experiments Circuit-simulation-based modeling of decoding power [Ganesan, Grover, Rabaey SiPS 11][Ganesan, Wen, Grover, Rabaey 12] log 10 (Pe) P T Shannon waterfall Regular LDPCs (3,4) (3,6) (4,8) (5,10) P total Power (Watts) R = 7 Gbps W = 7 GHz path-loss coeff. = 3 d = 3 meters fc = 60 GHz STM CMOS 90 nm Cadence Encounter (Layout) Synopsis HSPICE Bad code choice? 8/33

31 What can the total power look like? Some experiments Circuit-simulation-based modeling of decoding power [Ganesan, Grover, Rabaey SiPS 11][Ganesan, Wen, Grover, Rabaey 12] log 10 (Pe) P T Shannon waterfall Regular LDPCs (3,4) (3,6) (4,8) (5,10) Power (Watts) Bad code choice? P total Bad implementations? R = 7 Gbps W = 7 GHz path-loss coeff. = 3 d = 3 meters fc = 60 GHz STM CMOS 90 nm Cadence Encounter (Layout) Synopsis HSPICE 8/33

32 What can the total power look like? Some experiments Circuit-simulation-based modeling of decoding power [Ganesan, Grover, Rabaey SiPS 11][Ganesan, Wen, Grover, Rabaey 12] log 10 (Pe) P T Shannon waterfall Regular LDPCs (3,4) (3,6) (4,8) (5,10) Power (Watts) Bad code choice? Both? P total Bad implementations? R = 7 Gbps W = 7 GHz path-loss coeff. = 3 d = 3 meters fc = 60 GHz STM CMOS 90 nm Cadence Encounter (Layout) Synopsis HSPICE 8/33

33 What can the total power look like? Some experiments Circuit-simulation-based modeling of decoding power [Ganesan, Grover, Rabaey SiPS 11][Ganesan, Wen, Grover, Rabaey 12] log 10 (Pe) P T Shannon waterfall Regular LDPCs (3,4) (3,6) (4,8) (5,10) Power (Watts) Bad code choice? Both? P total Bad implementations? Neither? R = 7 Gbps W = 7 GHz path-loss coeff. = 3 d = 3 meters fc = 60 GHz STM CMOS 90 nm Cadence Encounter (Layout) Synopsis HSPICE 8/33

34 Fundamental limits on total power? M E P T D M 9/33

35 Fundamental limits on total power? M E P T D M log (P e ) 10 (Pe) fixed rate, distance Shannon limit (Tx power) Total power (Watts) Total power (watts) 9/33

36 Fundamental limits on total power? M E P T D M log (P e ) 10 (Pe) fixed rate, distance Shannon limit (Tx power) Total power (Watts) Total power (watts) 9/33

37 Fundamental limits on total power? M E P T D M 5 fixed rate, Total power behavior log (P e ) 10 (Pe) distance Shannon limit (Tx power) Total power (Watts) Total power (watts) 9/33

38 Fundamental limits on total power? M E P T D M log (P e ) 10 (Pe) fixed rate, distance Shannon limit (Tx power) Total power behavior this? Total power (Watts) Total power (watts) 9/33

39 Fundamental limits on total power? M E P T D M log (P e ) 10 (Pe) fixed rate, distance Shannon limit (Tx power) Total power behavior this?... or this? Total power (Watts) Total power (watts) 9/33

40 Fundamental limits on total power? M E P T D M Total power behavior 5 fixed rate, Questions for this talk: this? distance or this? 1) Must always diverge to infinity? log (P e ) 10 (Pe) P total 2) Should we operate close to capacity? Shannon limit (Tx power) 25 3) Can we use bounded? P T 30 4) What codes minimize? P total Total power (Watts) Total power (watts) 9/33

41 Our approach 10/33

42 Our approach Circuit Models 10/33

43 Comm. complexity Decentralized Estimation Our approach Circuit Models Network information theory Information flow in circuits 10/33

44 Comm. complexity Decentralized Estimation Our approach Circuit Models Network information theory Circuit power models 0 Information flow in circuits log 10 (Pe) Decoding power (Watts) Encoding/decoding power 10/33

45 Comm. complexity Decentralized Estimation Our approach Circuit Models Network information theory Circuit power models 0 Information flow in circuits log 10 (Pe) Decoding power (Watts) Encoding/decoding power log 10 (Pe) Shannon limit (Tx power) P total P T Tx power (Watts) Information theory (Transmit power) 10/33

46 Comm. complexity Decentralized Estimation Our approach Circuit Models Network information theory Circuit power models 0 Information flow in circuits log 10 (Pe) Decoding power (Watts) Encoding/decoding power log 10 (Pe) Shannon limit (Tx power) P total P T Tx power (Watts) Information theory (Transmit power) log 10 (Pe) P total 25 P T Ptotal x 10 5 Total Power (Watts) Fundamental bounds on total power 10/33

47 Obtaining fundamental limits: Main difficulty Code passing through channel Transmit power P T Models: AWGN fading... 11/33

48 Obtaining fundamental limits: Main difficulty Code passing through channel implementation (encoding/decoding) encoding/decoding power Models:?? Transmit power P T Models: AWGN fading... 11/33

49 Obtaining fundamental limits: Main difficulty Code passing through channel implementation (encoding/decoding) encoding/decoding power Models:?? Transmit power P T Models: AWGN fading... Need implementation models that are relevant and analyzable 11/33

50 A graphical model for circuit implementation M E P T D M Output nodes Helper nodes Input nodes VLSI model of computing [Thompson 80] - degree of each node at most α (e.g. 4) 12/33

51 A graphical model for circuit implementation M E P T D M Output nodes Helper nodes Input nodes VLSI model of computing [Thompson 80] - degree of each node at most α (e.g. 4) Channel model: Binary Symmetric Channel p ch p ch 1 p ch p ch p ch = Q ηpt N /33

52 Decoding complexity/power: Fundamental limits Theorem*[Grover, Woyach, Sahai ISIT 08, JSAC 11] # of clock-cycles, 1 log 1 τ log(α 1) log P e (C(P T ) R) 2 /V (P T )... for any code and any decoding algorithm. α : Max. node degree V (P T ) : channel dispersion * precise result for any P e and any gap appears in [Grover, Woyach, Sahai 11] 13/33

53 Decoding complexity: Proof Technique 14/33

54 Decoding complexity: Proof Technique Information-flow constrained by fan-out 14/33

55 Decoding complexity: Proof Technique Information-flow constrained by fan-out B i 14/33

56 Decoding complexity: Proof Technique Information-flow constrained by fan-out B i τ =1 14/33

57 Decoding complexity: Proof Technique Information-flow constrained by fan-out B i τ =1 τ =2 14/33

58 Decoding complexity: Proof Technique Information-flow constrained by fan-out B i τ =1 τ =2 τ =3 14/33

59 Decoding complexity: Proof Technique Information-flow constrained by fan-out B i τ =1 Sphere-packing for decoding neighborhoods [Grover 07][Grover, Sahai 08][Grover, Woyach, Sahai 11] τ =2 τ =3 14/33

60 Decoding complexity: Proof Technique Information-flow constrained by fan-out B i τ =1 Sphere-packing for decoding neighborhoods [Grover 07][Grover, Sahai 08][Grover, Woyach, Sahai 11] n nbd : number of nodes in decoding neighborhood τ =2 τ =3 14/33

61 P e e n nbd Decoding complexity: Proof Technique Information-flow constrained by fan-out Sphere-packing for decoding neighborhoods [Grover 07][Grover, Sahai 08][Grover, Woyach, Sahai 11] (C(P T ) R) 2 V (P T ) B i τ =1 n nbd : number of nodes in decoding neighborhood τ =2 τ =3 14/33

62 P e e n nbd Decoding complexity: Proof Technique Information-flow constrained by fan-out Sphere-packing for decoding neighborhoods [Grover 07][Grover, Sahai 08][Grover, Woyach, Sahai 11] (C(P T ) R) 2 V (P T ) τ B i τ =1 n nbd : number of nodes in decoding neighborhood 1 log(α 1) log(n nbd) τ =2 τ =3 14/33

63 P e e n nbd Decoding complexity: Proof Technique Information-flow constrained by fan-out Sphere-packing for decoding neighborhoods [Grover 07][Grover, Sahai 08][Grover, Woyach, Sahai 11] (C(P T ) R) 2 V (P T ) τ B i τ =1 n nbd : number of nodes in decoding neighborhood 1 log(α 1) log(n nbd) τ =2 τ =3 Theorem*[Grover, Woyach, Sahai ISIT 08, JSAC 11] # of clock-cycles,τ 1 log(α 1) log log 1 P e (C(P T ) R) 2 /V (P T ) 14/33

64 Node model: Fundamental bounds on total power Node model: each node consumes constant amount of energy per clock cycle 15/33

65 Node model: Fundamental bounds on total power Node model: each node consumes constant amount of energy per clock cycle P total =min P T P T + P decoding 15/33

66 Node model: Fundamental bounds on total power Node model: each node consumes constant amount of energy per clock cycle P total =min P T min P T P T + P decoding P T + γlog log 1 P e (C(P T ) R) 2 /V (P T ) 15/33

67 Node model: Fundamental bounds on total power Node model: each node consumes constant amount of energy per clock cycle P total =min P T min P T P T + P decoding P T + γlog log 1 P e (C(P T ) R) 2 /V (P T ) log 10 (Pe) Shannon limit (Tx power) P total P T Enode = 3 pj distance = 10 m fc = 60 GHz W = 3 GHz R = 1.5 Gbps Total power (Watts) 15/33

68 Node model: Fundamental bounds on total power Node model: each node consumes constant amount of energy per clock cycle P total =min P T min P T P T + P decoding P T + γlog log 1 P e (C(P T ) R) 2 /V (P T ) log 10 (Pe) Shannon limit (Tx power) P total P T Enode = 3 pj distance = 10 m fc = 60 GHz W = 3 GHz R = 1.5 Gbps Total power (Watts) 15/33

69 Node model: Fundamental bounds on total power Node model: each node consumes constant amount of energy per clock cycle P total =min P T min P T P T + P decoding P T + γlog log 1 P e (C(P T ) R) 2 /V (P T ) log 10 (Pe) Shannon limit (Tx power) P total P T Enode = 3 pj distance = 10 m fc = 60 GHz W = 3 GHz R = 1.5 Gbps Total power (Watts) 15/33

70 Node model: Fundamental bounds on total power Node model: each node consumes constant amount of energy per clock cycle P total =min P T min P T P T + P decoding P T + γlog log 1 P e (C(P T ) R) 2 /V (P T ) log 10 (Pe) Shannon limit (Tx power) P total P T Enode = 3 pj distance = 10 m fc = 60 GHz W = 3 GHz R = 1.5 Gbps Total power (Watts) Moral: do not operate too close to capacity! 15/33

71 Node model What codes are good? 16/33

72 Node model What codes are good? Capacity-approaching LDPCs, iterative decoding? 16/33

73 Node model What codes are good? Capacity-approaching LDPCs, iterative decoding? Random block codes, ML decoding? 16/33

74 Node model What codes are good? Capacity-approaching LDPCs, iterative decoding? Convolutional codes, Viterbi decoding Random block codes, ML decoding? 16/33

75 Node model What codes are good? Capacity-approaching LDPCs, iterative decoding? Convolutional codes, Viterbi decoding Random block codes, ML decoding? Polar codes, successive cancelation? 16/33

76 Node model What codes are good? Capacity-approaching LDPCs, iterative decoding? Convolutional codes, Viterbi decoding Random block codes, ML decoding? Polar codes, successive cancelation? 16/33

77 Node model What codes are good? Capacity-approaching LDPCs, iterative decoding? Convolutional codes, Viterbi decoding Random block codes, ML decoding? Polar codes, successive cancelation? Regular LDPCs, iterative decoding [Lentmaier 05][Grover, Woyach, Sahai JSAC 11] 16/33

78 Node model What codes are good? Capacity-approaching LDPCs, iterative decoding? Convolutional codes, Viterbi decoding Random block codes, ML decoding? Polar codes, successive cancelation? Regular LDPCs, iterative decoding [Lentmaier 05][Grover, Woyach, Sahai JSAC 11] db 25 log 10 (Pe) Optimizing transmit power for (3,4) LDPC Lower bound on total power Total power for (3,4) LDPC, Gallager decoding 125 Shannon limit !"#$%&'"()*&+,$##-.&%"/ -0$%)1 Total power (Watts) 16/33

79 Node model What codes are good? Capacity-approaching LDPCs, iterative decoding? Convolutional codes, Viterbi decoding Random block codes, ML decoding? Polar codes, successive cancelation? Regular LDPCs, iterative decoding [Lentmaier 05][Grover, Woyach, Sahai JSAC 11] db 25 log 10 (Pe) Optimizing transmit power for (3,4) LDPC Lower bound on total power Total power for (3,4) LDPC, Gallager decoding Data-centers (10 GBASE-T): regular LDPC! 125 Shannon limit !"#$%&'"()*&+,$##-.&%"/ -0$%)1 Total power (Watts) 16/33

80 Talk outline Information flows: visible and invisible M E P T D M Traditional: Minimize P T Now: Minimize total (transmit + proc.) power Fundamental limits Node model: stay away from capacity regular LDPCs are order optimal 17/33

81 Talk outline Information flows: visible and invisible M E P T D M Traditional: Minimize P T Now: Minimize total (transmit + proc.) power Fundamental limits Node model: stay away from capacity regular LDPCs are order optimal Wire model: using bounded P T suboptimal 17/33

82 What did we expect from our experimental results? Experiment-based modeling of decoding power [Ganesan, Grover, Rabaey SiPS 11][Ganesan, Wen, Grover, Rabaey 12] P T Regular LDPCs (3,4) (3,6) (4,8) (5,10) P total R = 7 Gbps W = 7 GHz path-loss coefft = 3 d = 3 meters fc = 60 GHz STM CMOS 90 nm Cadence Encnter (Layout) Synopsis HSPICE /33

83 Wire model of power consumption 19/33

84 Wire model of power consumption 19/33

85 Wire model of power consumption Fourier Transform [Thompson 80] Power A wiresτ n = Ω n log n Similar results for sorting, boolean functions,... for implementing any algorithm. 19/33

86 Wire model of power consumption Fourier Transform [Thompson 80] Power A wiresτ n = Ω n log n Similar results for sorting, boolean functions,... for implementing any algorithm. Encoding/decoding complexity: [El Gamal et al. 84] A chip τ 2 Ω nr 2 log npe 19/33

87 Wire model: Fundamental limits on encoding/decoding power Theorem *[Grover, Goldsmith, Sahai Allerton 11; ISIT Sub. 12] log 1 Encoding/decoding Power Ω P e P T... for any code and any encoding/decoding algorithm. * precise result for any P e and any gap appears in [Grover, Goldsmith, Sahai 12] 20/33

88 Wire model: Fundamental limits on on total power P total =min P T P T + P decoding +P encoding =minp T +2β P T log 1 P e P T 21/33

89 Wire model: Fundamental limits on on total power P total =min P T P T + P decoding +P encoding =minp T +2β P T log 1 P e P T 5 log 10 (Pe) P total d = 0.5 m path-loss coeff = 2 Capacitance = 1 ff/um CMOS 90 nm tech x 10 5 Power (Watts) 21/33

90 Wire model: Fundamental limits on on total power P total =min P T P T + P decoding +P encoding =minp T +2β P T log 1 P e P T 5 log 10 (Pe) P T P total d = 0.5 m path-loss coeff = 2 Capacitance = 1 ff/um CMOS 90 nm tech x 10 5 Power (Watts) 21/33

91 Wire model: Fundamental limits on on total power P total =min P T P T + P decoding +P encoding =minp T +2β P T log 1 P e P T log 10 (Pe) P T P total 3 log 1 P e x 10 5 Power (Watts) d = 0.5 m path-loss coeff = 2 Capacitance = 1 ff/um CMOS 90 nm tech. 21/33

92 Wire model: Fundamental limits on on total power P total =min P T P T + P decoding +P encoding =minp T +2β P T log 1 P e P T log 10 (Pe) P T P total 3 log 1 P e x 10 5 Power (Watts) P total with bounded P T d = 0.5 m path-loss coeff = 2 Capacitance = 1 ff/um CMOS 90 nm tech. 21/33

93 Wire model: Fundamental limits on on total power P total =min P T P T + P decoding +P encoding =minp T +2β P T log 1 P e P T log 10 (Pe) P T P total 3 log 1 P e x 10 5 Power (Watts) P total with bounded P T log 1 P e d = 0.5 m path-loss coeff = 2 Capacitance = 1 ff/um CMOS 90 nm tech. 21/33

94 Wire model: Fundamental limits on on total power P total =min P T P T + P decoding +P encoding =minp T +2β P T log 1 P e P T log 10 (Pe) P T P total 3 log 1 P e x 10 5 Power (Watts) P total with bounded P T log 1 P e d = 0.5 m path-loss coeff = 2 Capacitance = 1 ff/um CMOS 90 nm tech. Moral: Bounded P T is suboptimal 21/33

95 Suboptimality of bounded transmit power: Experimental results 22/33

96 Information flows in circuits: Summary M E D M P T Information needs to flow at the encoder and decoder (mixing) (de-mixing) Fundamental limits: Traditional: Operate close to capacity Total power perspective: Node model: stay away from capacity Wire model: use unbounded P T No assumptions on the code structure 23/33

97 Looking forward: Greenest codes for wireless? Grand goal: Code/encoder/decoder triples within bounded dbs of the optimal. 5 log (P ) 10 (Pe) e Shannon limit (Tx power) Total power (Watts) Transmit Total power power (Watts) (watts) 24/33

98 log 10 (Pe) Looking forward: Greenest codes for wireless? Grand goal: Code/encoder/decoder triples within bounded dbs of the optimal. Distance (meters) /33

99 log 10 (Pe) Looking forward: Greenest codes for wireless? Grand goal: Code/encoder/decoder triples within bounded dbs of the optimal. 5-5 (3,6), girth-6, 1-bit msg Distance (meters) (4,8), girth-6, 1-bit msg (3,6), girth-6, 2-bit msg (3,6), girth-8, 1-bit msg (3,6)-LDPC girth-8, 2-bit msg R = 7 Gbps W = 7 GHz path-loss coeff = 3 fc = 60 GHz STM CMOS 90 nm Layout:Cadence Enctr Synopsis HSPICE [Ganesan, Grover, Rabaey 11][Ganesan, Wen, Grover, Rabaey, Goldsmith 12] 24/33

100 log 10 (Pe) Looking forward: Greenest codes for wireless? Grand goal: Code/encoder/decoder triples within bounded dbs of the optimal. 5-5 (3,6), Uncoded girth-6, transmission (4,8), girth-6, 1-bit msg 1-bit msg Distance (meters) (3,6), girth-6, 2-bit msg (3,6), girth-8, 1-bit msg (3,6)-LDPC girth-8, 2-bit msg R = 7 Gbps W = 7 GHz path-loss coeff = 3 fc = 60 GHz STM CMOS 90 nm Layout:Cadence Enctr Synopsis HSPICE [Ganesan, Grover, Rabaey 11][Ganesan, Wen, Grover, Rabaey, Goldsmith 12] 24/33

101 Looking forward: Green data-center communication !2 10!4 FER/BER log 10 (Pe) Traditional view Code choice (10-Gbase-T): (6,32) - RS-LDPC, length 2048 Total-power perspective Distance (meters) FER/BER 10!6 10!8 10!10 10!12 10!14 10 iterations 20 iterations 50 iterations 100 iterations 200 iterations Eb/No (db) SNR (db) /33

102 Looking forward: Green data-center communication !2 10!4 FER/BER log 10 (Pe) Traditional view Code choice (10-Gbase-T): (6,32) - RS-LDPC, length 2048 Total-power perspective Distance (meters) FER/BER 10!6 10!8 10!10 10!12 10!14 10 iterations 20 iterations 50 iterations 100 iterations 200 iterations Eb/No (db) SNR (db) (6,32) - RS-LDPC length /33

103 Looking forward: Green data-center communication !2 10!4 FER/BER log 10 (Pe) Traditional view Code choice (10-Gbase-T): (6,32) - RS-LDPC, length 2048 Total-power perspective Distance (meters) FER/BER 10!6 10!8 10!10 10!12 10!14 10 iterations 20 iterations 50 iterations 100 iterations 200 iterations Eb/No (db) SNR (db) /33

104 Looking forward: Green data-center communication !2 10!4 FER/BER log 10 (Pe) Traditional view Code choice (10-Gbase-T): (6,32) - RS-LDPC, length 2048 Total-power perspective Distance (meters) FER/BER 10!6 10!8 10!10 10!12 10!14 10 iterations 20 iterations 50 iterations 100 iterations 200 iterations Eb/No (db) SNR (db) Code-choice must depend on distance 25/33

105 Looking forward: Energy costs of communication 26/33

106 Looking forward: Energy costs of communication Other models of computing - e.g. nodes/wires can sleep 26/33

107 Looking forward: Energy costs of communication Other models of computing - e.g. nodes/wires can sleep Multiple simultaneously transmitting links - Code-choice must depend on distance, density [Grover, Woyach, Sahai JSAC 11] Tx r Rx θ d 26/33

108 Looking forward: Energy costs of communication Other models of computing - e.g. nodes/wires can sleep Multiple simultaneously transmitting links - Code-choice must depend on distance, density [Grover, Woyach, Sahai JSAC 11] Tx r Rx θ d Other components of system power - Equalizer [Grover, Sahai, Park 11] M Encoder PA LNA ADC Equalizer Decoder M 26/33

109 Talk outline 1) Information flows in circuits M Transmitter P T Channel Receiver Communicating and processing information 2) Information flows in physical systems M Communicating and Using information C w1 C w 2 Implicit communication C w 6 Implicit channel Plant C w 3 Witsenhausen s counterexample - approximately optimal solutions C w 5 C w 4 Beyond the counterexample 27/33

110 Looking forward: Implicit information flows in control C w1 C w 2 C w 6 Plant C w 3 C w 5 C w 4 28/33

111 Looking forward: Implicit information flows in control M E P T D Communication theory M C w1 C w 2 C w 6 Plant C w 3 C w 5 C w 4 28/33

112 Looking forward: Implicit information flows in control M E P T D Communication theory M C w1 C w 2 C w 6 Plant C w 3 C w 5 C w 4 28/33

113 Looking forward: Implicit information flows in control M E P T D Communication theory M C w1 C w 2 u Plant Control agent y Control theory C w 6 Plant C w 3 C w 5 C w 4 28/33

114 Looking forward: Implicit information flows in control M E P T D Communication theory M C w1 C w 2 u Plant Control agent y Control theory C w 6 Plant C w 3 C w 5 C w 4 28/33

115 Looking forward: Implicit information flows in control M E P T D Communication theory M C w1 C w 2 u Plant Control agent y Control theory C w 6 Plant C w 3 C w 5 C w 4... semantic aspects of communication are irrelevant to the engineering problem. [Shannon 48] 28/33

116 Looking forward: Implicit information flows in control M E P T D Communication theory M C w1 C w 2 u Plant Control agent y Control theory C w 6 Plant C w 3 C w 5 C w 4 [comm theory] deals with an essentially simple problem, because the transmission of information is considered independently of its use. [Witsenhausen 71] 28/33

117 Looking forward: Implicit information flows in control M E P T D Communication theory M C w1 C w 2 u Plant Control agent y Control theory C w 6 Plant C w 3 C w 5 C w 4 28/33

118 Looking forward: Implicit information flows in control M E P T D Communication theory M C w1 C w 2 u Plant Control agent y Control theory Message? C w 6 Plant C w 3 C w 5 C w 4 28/33

119 Looking forward: Implicit information flows in control M E P T D Communication theory M C w1 C w 2 u Plant Control agent y Control theory Message? a choice C w 6 Plant C w 3 C w 5 C w 4 28/33

120 Looking forward: Implicit information flows in control M E P T D Communication theory M C w1 C w 2 u Plant Control agent y Control theory Message? a choice C w 6 Implicit channel Plant C w 3 C w 5 C w 4 28/33

121 Looking forward: Implicit information flows in control M E P T D Communication theory M C w1 C w 2 u Plant Control agent y Control theory Message? a choice C w 6 Implicit channel Plant C w 5 C w 4 C w 3 Dual role of actions control communicate 28/33

122 Looking forward: Implicit information flows in control M E P T D Communication theory M C w1 C w 2 u Plant Control agent y Control theory Message? a choice C w 6 Implicit channel Plant C w 5 C w 4 C w 3 Dual role of actions control communicate x 1 x 2 + x C u 1 + C 1 2 u 2 z y N (0, 1) 28/33

123 Looking forward: Implicit information flows in control M E P T D Communication theory M C w1 C w 2 u Plant Control agent y Control theory Message? a choice C w 6 Implicit channel Plant C w 5 C w 4 C w 3 Dual role of actions control communicate x 1 x 2 + x C u 1 + C 1 2 u 2 z y N (0, 1) 28/33

124 Breakdown of traditional theory: Witsenhausen s counterexample x 1 x 2 + x C u 1 + C 1 2 u 2 z y N (0, 1) x 0 N (0, σ0) 2 min k 2 E u E x 2 2 Linear, Quadratic, Gaussian [Witsenhausen 68] 29/33

125 Breakdown of traditional theory: Witsenhausen s counterexample x 1 x 2 + x C u 1 + C 1 2 u 2 z y N (0, 1) x 0 N (0, σ0) 2 min k 2 E u E x 2 2 Linear, Quadratic, Gaussian [Witsenhausen 68] 29/33

126 Breakdown of traditional theory: Witsenhausen s counterexample x 1 x 2 + x C u 1 + C 1 2 u 2 z y N (0, 1) x 0 N (0, σ0) 2 min k 2 E u E x 2 2 Linear, Quadratic, Gaussian [Witsenhausen 68] 29/33

127 Breakdown of traditional theory: Witsenhausen s counterexample x 1 x 2 + x C u 1 + C 1 2 u 2 z y N (0, 1) x 0 N (0, σ0) 2 min k 2 E u E x 2 2 Linear, Quadratic, Gaussian [Witsenhausen 68] Nonlinear strategies can outperform linear strategies [Witsenhausen 68] 29/33

128 Breakdown of traditional theory: Witsenhausen s counterexample x 1 x 2 + x C u 1 + C 1 2 u 2 z y N (0, 1) x 0 N (0, σ0) 2 min k 2 E u E x 2 2 Linear, Quadratic, Gaussian [Witsenhausen 68] Nonlinear strategies can outperform linear strategies [Witsenhausen 68] Nonconvex [Witsenhausen 68], Intractable (NP-hard) [Papadimitriou, Tsitsiklis 86] 29/33

129 Breakdown of traditional theory: Witsenhausen s counterexample x 1 x 2 + x C u 1 + C 1 2 u 2 z y N (0, 1) x 0 N (0, σ0) 2 min k 2 E u E x 2 2 Linear, Quadratic, Gaussian [Witsenhausen 68] Nonlinear strategies can outperform linear strategies [Witsenhausen 68] Nonconvex [Witsenhausen 68], Intractable (NP-hard) [Papadimitriou, Tsitsiklis 86] Semi-exhaustive searches [Deng, Ho 99] [Baglietto et al. 01] [Lee, Lau, Ho 01][Li, Marden, Shamma 09] [Karlsson et al. 10] 29/33

130 Breakdown of traditional theory: Witsenhausen s counterexample x 1 x 2 + x C u 1 + C 1 2 u 2 z y N (0, 1) x 0 N (0, σ0) 2 min k 2 E u E x 2 2 Linear, Quadratic, Gaussian Nonlinear strategies can outperform linear strategies [Witsenhausen 68] Nonconvex [Witsenhausen 68], Intractable (NP-hard) [Papadimitriou, Tsitsiklis 86] Semi-exhaustive searches [Deng, Ho 99] [Baglietto et al. 01] [Lee, Lau, Ho 01][Li, Marden, Shamma 09] [Karlsson et al. 10] x 1 x 1 [Witsenhausen 68] x 0 x 0 29/33

131 Our approach: understanding information flows to obtain fundamental limits on cost 30/33

132 Our approach: understanding information flows to obtain fundamental limits on cost x 1 x 2 + x u 1 + u 2 C 1 C 2 z y N (0, 1) Original problem [Witsenhausen 68] 30/33

133 Our approach: understanding information flows to obtain fundamental limits on cost x 1 x 2 + x u 1 + u 2 C 1 C 2 z y N (0, 1) Original problem [Witsenhausen 68] E u 1 x 0 + z x 1 y x 1 + D Identify information-flow [Grover, Sahai 08] 30/33

134 Our approach: understanding information flows to obtain fundamental limits on cost x 1 x 2 + x u 1 + u 2 C 1 C 2 z y N (0, 1) Original problem [Witsenhausen 68] E u 1 x 0 + z x 1 y x 1 + D E b 1 b 2.b 3 b 4 b 5 x 0 x 1 y + u + D 1 b 1 b 2.b 3 b 4 b 5 b 1 b 2. z 0.z 1 z 2 z 3 u 2 = x 1 Understand information-flow: Binary version [Avestimehr et al. 08][Grover, Sahai 09] Identify information-flow [Grover, Sahai 08] 30/33

135 Our approach: understanding information flows to obtain fundamental limits on cost x 1 x 2 + x u 1 + u 2 C 1 C 2 z y N (0, 1) x m C 1 C 2 u m 1 x m 1 z m u m 2 x m 2 Original problem [Witsenhausen 68] E u 1 x 0 + z x 1 y x 1 + D Asymptotic vector version [Grover, Sahai 08] E b 1 b 2.b 3 b 4 b 5 x 0 x 1 y + u + D 1 b 1 b 2.b 3 b 4 b 5 b 1 b 2. z 0.z 1 z 2 z 3 u 2 = x 1 Understand information-flow: Binary version [Avestimehr et al. 08][Grover, Sahai 09] Identify information-flow [Grover, Sahai 08] 30/33

136 Our approach: understanding information flows to obtain fundamental limits on cost Large-deviations/ sphere-packing [Grover, Park, Sahai 09, 12] x 1 x 2 + x u 1 + u 2 C 1 C 2 z y N (0, 1) x m C 1 C 2 u m 1 x m 1 z m u m 2 x m 2 Original problem [Witsenhausen 68] E u 1 x 0 + z x 1 y x 1 + D Asymptotic vector version [Grover, Sahai 08] E b 1 b 2.b 3 b 4 b 5 x 0 x 1 y + u + D 1 b 1 b 2.b 3 b 4 b 5 b 1 b 2. z 0.z 1 z 2 z 3 u 2 = x 1 Understand information-flow: Binary version [Avestimehr et al. 08][Grover, Sahai 09] Identify information-flow [Grover, Sahai 08] 30/33

137 Approx. solutions to Witsenhausen s counterexample x 1 x 2 + x u 1 + u 2 C 1 C 2 z y N (0, 1) min k 2 E u E x 2 2 Theorem [Grover, Park, Sahai IEEE Tran Auto Cont 12] For all (k, σ 0 ), Upper bound Lower bound 8 Upper bound Scalar quantization Lower bound Info-theory + Large deviations First approximately-optimal solutions to Witsenhausen s counterexample 31/33

138 Approx. solutions to Witsenhausen s counterexample min k 2 E u E x 2 2 Upper bound x 1 x 2 + x u 1 + u 2 C 1 C 2 z y N (0, 1) Scalar quantization Theorem [Grover, Park, Sahai IEEE Tran Auto Cont 12] For all (k, σ 0 ), x 1 Watermarking ( dirty-paper coding) Upper bound Lower bound 8 Lower bound Info-theory + Large deviations First approximately-optimal solutions to Witsenhausen s counterexample x 0 31/33

139 Looking forward: Information-flows in Cyber-Physical Systems 32/33

140 Looking forward: Information-flows in Cyber-Physical Systems C2 C w w1 y 1 y 2 u 2 u 1 System 32/33

141 Looking forward: Information-flows in Cyber-Physical Systems C2 C w w1 y 1 u 2 System 32/33

142 Looking forward: Information-flows in Cyber-Physical Systems External channel C2 C w w1 y 1 u 2 System 32/33

143 Looking forward: Information-flows in Cyber-Physical Systems External channel C2 C w w1 y 1 sensor u 2 System 32/33

144 Looking forward: Information-flows in Cyber-Physical Systems External channel C2 C w w1 y 1 sensor u 2 actuator System 32/33

145 Looking forward: Information-flows in Cyber-Physical Systems External channel C2 C w w1 y 1 sensor u 2 actuator System 32/33

146 Looking forward: Information-flows in Cyber-Physical Systems External channel C2 C w w1 y 1 sensor u 2 actuator System 32/33

147 Looking forward: Information-flows in Cyber-Physical Systems External channel C2 C w w1 y 1 sensor u 2 actuator System ν t + Unit delay x t z t C 1 + C s 2 x t sensor t actuator Communicating a evolving state [Bansal, Basar 89][Wong, Brockett 90][Tatikonda, Mitter 01] [Sahai, Mitter 06] [Sinopoli et al. 04]... 32/33

148 Looking forward: Information-flows in Cyber-Physical Systems External channel C2 C w w1 y 1 y 2 sensor u 1 u 2 System actuator ν t + Unit delay x t z t C 1 + C s 2 x t sensor t actuator Communicating a evolving state [Bansal, Basar 89][Wong, Brockett 90][Tatikonda, Mitter 01] [Sahai, Mitter 06] [Sinopoli et al. 04]... 32/33

149 Looking forward: Information-flows in Cyber-Physical Systems External channel C2 C w w1 y 1 y 2 sensor u 1 u 2 System actuator ν t + Unit delay x t z t C 1 + C s 2 x t sensor t actuator Communicating a evolving state [Bansal, Basar 89][Wong, Brockett 90][Tatikonda, Mitter 01] [Sahai, Mitter 06] [Sinopoli et al. 04]... ν t + x t Unit Delay + x t x C 1 + t C 2 u 1 z [Grover, Sahai 10] R ex [Grover 12] 32/33

150 Looking forward: Information-flows in Cyber-Physical Systems External channel C2 C w w1 y 1 y 2 sensor u 1 u 2 System actuator ν t + Unit delay x t z t C 1 + C s 2 x t sensor t actuator Communicating a evolving state [Bansal, Basar 89][Wong, Brockett 90][Tatikonda, Mitter 01] [Sahai, Mitter 06] [Sinopoli et al. 04]... ν t + x t Unit Delay + x t x C 1 + t C 2 u 1 z [Grover, Sahai 10] R ex [Grover 12] 32/33

151 Summary: Understanding hidden information-flows is useful Information flows in circuits Information flows in physical systems M E P T D M C w1 C w2 C w6 Implicit channel Plant C w3 C w5 C w4 Information flow in computing: fundamental bounds on total power Implicit information flows: approximately-optimal solutions 33/33

152 Backup slides 34/33

153 35/33

154 The appropriate approximations in the two bounds log 10 (Pe) Shannon limit (Tx power) P total P T Node model log 10 (Pe) P total Wire model Total power (Watts) Total power (Watts) x 10 5 Total Power power (Watts) (Watts) Dispersion approximation E sp (R) Sphere-packing exponent Approximation far from capacity 0 R C 36/33

155 Total power minimizing density Density (links/m 2 ) Optimal code (if decoding were free) r = 1m θ = 0 W = 3 GHz α = 3 P e = R = 1.5 Gbps data Outer bound with E node = 3 pj Uncoded transmission Extrapolated performance of the decoder of [Zhang et al.] Total Power (watts) 37/33

156 Wireless information transfer i v s s r s M i 2 Load l L 1 L r 2 l c l i 38/33

157 Wireless information transfer i v s s r s M i 2 Load l L 1 L r 2 l c l i 38/33

158 Wireless information transfer i v s s r s M i 2 Load l L 1 L r 2 l c l i z + v o + - v 38/33

159 Optimal information transfer strategy? 39/33

160 Optimal information transfer strategy? N(f)/η(f) Shannon strategy : waterfilling frequency (Hz) in log scale 39/33

161 Optimal information transfer strategy? C(P avail ) = max P 1,...,P n s.t. np P i P avail i=1 n i=1 log ( 1+ η ip i N ) N(f)/η(f) Shannon strategy : waterfilling frequency (Hz) in log scale 39/33

162 Optimal information transfer strategy? C(P avail ) = max P 1,...,P n s.t. np P i P avail i=1 n i=1 log ( 1+ η ip i N ) N(f)/η(f) Shannon strategy : waterfilling frequency (Hz) in log scale 39/33

163 Secure Communication Using Inductive Coupling!"#$%&'()*%+$,-'".*!"#$%&"'"()*+&,-"$$*-#%$"$*.")%(/(0* /$0"&$1*)"&0".-)1)"&*!"#$%&"'"()*+&,-"$$*-#%$"$* 1#2"3,&'*-,44#+$"* 5#2"$.&,++"&*.")"-)/,(*)6&,%06* $+"-)&%'*$"($/(0* 5#2"$.&,++"&*.")"-)/,(*)6&,%06* +67$/-$* 8(3,&'#)/,(*."$)&%-)/,(*)6&,%06* -#+#-/)79#++&,#-6/(0*-,."$*#(.*$)&,(0* -,(2"&$"*,3*-6#(("4*-,./(0*)6",&"'*!"#$%&'()*+),*#'-&.'/$&'0"* 8(3,&'#)/,(*."$)&%-)/,(*)6&,%06* +67$/-$* :%#()%'*;"7*./$)&/<%)/,(* 40/33

164 A deterministic abstraction x 0 + x 1 + x x 0 z E R ex + z N (0, 1) D u 1 u 2 = x 1 k 2 E u + E (x 2 ) 2 1 x 1 x 1 E u 1 u 2 + x 1 R ex + D 41/33

165 A deterministic abstraction x 0 + x 1 + x x 0 z E R ex + z N (0, 1) D u 1 u 2 = x 1 k 2 E u + E (x 2 ) 2 1 x 1 x 1 E u 1 u 2 + x 1 R ex + D x 0 b 1 b 2 b 3 b 4 b 5 41/33

166 A deterministic abstraction x 0 + x 1 + x x 0 z E R ex + z N (0, 1) D u 1 u 2 = x 1 k 2 E u + E (x 2 ) 2 1 x 1 x 1 E u 1 u 2 + x 1 R ex + D b 1 b 2 x x 1 0 D x 1 b 3 b 4 b 5 41/33

167 A deterministic abstraction x 0 + x 1 + x x 0 z E R ex + z N (0, 1) D u 1 u 2 = x 1 k 2 E u + E (x 2 ) 2 1 x 1 x 1 E u 1 u 2 + x 1 R ex + D b 1 b 2 x x 1 0 D x 1 b 3 b 4 b 5 z 2 41/33

168 A deterministic abstraction x 0 + x 1 + x x 0 z E R ex + z N (0, 1) D u 1 u 2 = x 1 k 2 E u + E (x 2 ) 2 1 x 1 x 1 E u 1 u 2 + x 1 R ex + D x 0 b 1 b 2 b 3 b 4 b 5 x 1 D x 1 E z 2 41/33

169 A deterministic abstraction x 0 + x 1 + x x 0 z E R ex + z N (0, 1) D u 1 u 2 = x 1 k 2 E u + E (x 2 ) 2 1 x 1 x 1 E u 1 u 2 + x 1 R ex + D x 0 b 1 b 2 b 3 b 4 b 5 x 1 D x 1 E z 2 R ex = 2 41/33

170 A deterministic abstraction x 0 + x 1 + x x 0 z E R ex + z N (0, 1) D u 1 u 2 = x 1 k 2 E u + E (x 2 ) 2 1 x 1 x 1 E u 1 u 2 + x 1 R ex + D x 0 b 1 b 2 b 3 b 4 b 5 x 1 E u 1 z 2 D x 1 R ex = 2 41/33

171 Strategy for deterministic abstraction b 1 b 2 b 3 b 4 b 5 x 1 x 0 D x 1 E u 1 z 2 R ex = 2 42/33

172 Strategy for deterministic abstraction b 1 b 2 b 3 b 4 b 5 x 1 x 0 D x 1 E u 1 z 2 R ex = 2 42/33

173 Strategy for deterministic abstraction b 1 b 2 b 3 b 4 b 5 x 1 x 0 D x 1 E u 1 z 2 R ex = 2 42/33

174 Strategy for deterministic abstraction b 1 b 2 b 3 b 4 b 5 x 1 x 0 b 1 b 2 D x 1 E u 1 z 2 R ex = 2 42/33

175 Strategy for deterministic abstraction b 1 b 2 b 3 b 4 b 5 x 1 x 0 b 1 b 2 D x 1 E u 1 z 2 R ex = 2 42/33

176 Strategy for deterministic abstraction b 1 b 2 b 3 b 4 b 5 x 1 x 0 b 1 b 2 D x 1 E u 1 z 2 R ex = 2 42/33

177 Strategy for deterministic abstraction b 1 b 2 b 3 b 4 b 5 x 1 x 0 b 1 b 2 D x 1 E u 1 z 2 R ex = 2 b 3 b 4 42/33

178 Strategy for deterministic abstraction b 1 b 2 b 3 b 4 b 5 x 1 x 0 b 1 b 2 D x 1 b 3 b 4 E u 1 z 2 R ex = 2 b 3 b 4 42/33

179 Strategy for deterministic abstraction b 1 b 2 b 3 b 4 b 5 x 1 x 0 b 1 b 2 D x 1 b 3 b 4 E u 1 z 2 R ex = 2 b 3 b 4 42/33

180 Strategy for deterministic abstraction b 1 b 2 b 3 b 4 b 5 x 1 x 0 b 1 b 2 D x 1 b 3 b 4 E u 1 b 5 b 3 b 4 z 2 R ex = 2 42/33

181 Strategy for deterministic abstraction b 1 b 2 b 3 b 4 b 5 E x 1 x 0 b 1 b 2 D x 1 u 1 z 2 R ex = 2 b 3 b 4 0 b 5 b 3 b 4 42/33

182 Strategy for deterministic abstraction b 1 b 2 b 3 b 4 b 5 E x 1 x 0 b 1 b 2 D x 1 u 1 z 2 R ex = 2 b 3 b 4 0 b 5 b 3 b 4 42/33

183 Strategy for deterministic abstraction b 1 b 2 b 3 b 4 b 5 E x 1 x 0 b 1 b 2 D x 1 u 1 z 2 R ex = 2 b 3 b 4 0 b 5 b 3 b 4 42/33

184 Strategy for deterministic abstraction b 1 b 2 b 3 b 4 b 5 E x 1 x 0 b 1 b 2 D x 1 u 1 z 2 R ex = 2 b 3 b 4 0 b 5 b 3 b 4 Explicit channel: fine information 42/33

185 Strategy for deterministic abstraction b 1 b 2 b 3 b 4 b 5 E x 1 x 0 b 1 b 2 D x 1 u 1 z 2 R ex = 2 b 3 b 4 0 b 5 b 3 b 4 Explicit channel: fine information 42/33

186 Strategy for deterministic abstraction b 1 b 2 b 3 b 4 b 5 E x 1 x 0 b 1 b 2 D x 1 u 1 z 2 R ex = 2 b 3 b 4 0 b 5 b 3 b 4 Explicit channel: fine information 42/33

187 Strategy for deterministic abstraction b 1 b 2 b 3 b 4 b 5 E x 1 x 0 b 1 b 2 D x 1 u 1 z 2 R ex = 2 b 3 b 4 0 b 5 b 3 b 4 Explicit channel: fine information Implicit channel: high order bits 42/33

188 Strategy for deterministic abstraction b 1 b 2 b 3 b 4 b 5 E x 1 x 0 b 1 b 2 D x 1 u 1 z 2 R ex = 2 b 3 b 4 0 b 5 b 3 b 4 Explicit channel: fine information Implicit channel: high order bits 42/33

189 Strategy for deterministic abstraction b 1 b 2 b 3 b 4 b 5 E x 1 x 0 b 1 b 2 D x 1 u 1 z 2 R ex = 2 b 3 b 4 0 b 5 b 3 b 4 Explicit channel: fine information Implicit channel: high order bits binning! 42/33

190 Infinitely better than the natural strategies!!" # Nonlinear +,-./-0123/45./6/7>3 implicit,./ /6/73?@127/-ab linear explicit [Martins '06] " 3:3; 6< =33:3!" # SNR ex = P ex = σ 2 0 k = 10 2 Average cost cost!" $ +,-./-0123/45./6/ /6/7 Our synergistic strategy ratio diverges to infinity!!"" #"" $"" %"" &"" '"" ("" )"" *"" σ 0" 43/33

191 Is the counterexample relevant? x C u 1 + C 1 2 u 2 z + x 2 min max x 0,z k 2 u x 2 2 x z2 Induced norm: linear strategies optimal! [Rotkowitz 06] Witsenhausen s LQG formulation: more assumptions harder! Q. Why consider the quadratic norm? [John Doyle, Paths ahead, MIT, 2009] 44/33

192 Induced norm formulation too conservative? + x 2 x u 1 + u 2 C 1 C 2 z min max x 0,z k 2 u x 2 2 x z2 Makes no assumptions: noise completely arbitrary, adversarial! Usually we know something about the noise, e.g. typical values, or bounds + x 2 x u 1 + u 2 C 1 C 2 z ( a, a) say z ( a, a) A quadratic cost min max x 0,z ( a,a) k2 u x 2 2 Quantization (signaling) : can outperform linear by an unbounded factor is approximately optimal! 45/33

193 Induced norm Which is a right norm? + x 2 x u 1 + u 2 C 1 C 2 z Quadratic norm min max x 0,z k 2 u x 2 2 x z2 min max x 0,z ( a,a) k2 u x 2 2 Induced norm: cost normalized by size of state and noise. In many practical cases, small perturbations and small noises do cost less! Quadratic norm captures how much large perturbations/noises hurt Moral: signal even in adversarial situations 46/33