Universal Anytime Codes: An approach to uncertain channels in control

Universal Anytime Codes: An approach to uncertain channels in control paper by Stark Draper and Anant Sahai presented by Sekhar Tatikonda Wireless Foundations Department of Electrical Engineering and Computer Sciences UC Berkeley Mitsubishi Electric Research Labs Cambridge, MA, USA April 2, 27 Draper,Sahai (UC Berkeley) Universal Anytime ConCom7 / 23

Outline Problem setup Channel uncertainty and stabilization Review of past results 2 New result: universal anytime codes 3 Sufficient condition for stabilization 4 Conclusion Draper,Sahai (UC Berkeley) Universal Anytime ConCom7 2 / 23

Our simple distributed control problem U t Step Delay D t Unstable System X t Designed Observer O Possible Control Knowledge U t Control Signals Designed Controller C Possible Channel Feedback Delay Uncertain Channel W Step X t+ = λx t + U t + D t Unstable λ >, bounded initial condition and disturbance D Draper,Sahai (UC Berkeley) Universal Anytime ConCom7 4 / 23

Our simple distributed control problem U t Step Delay D t Unstable System X t Designed Observer O Possible Control Knowledge U t Control Signals Designed Controller C Possible Channel Feedback Delay Uncertain Channel W Step X t+ = λx t + U t + D t Unstable λ >, bounded initial condition and disturbance D Goal: η-stabilization sup t> E[ X t η ] K for some K < Draper,Sahai (UC Berkeley) Universal Anytime ConCom7 4 / 23

Model of Channel Uncertainty: Compound Channels Channel W is known to be memoryless across time Input and output alphabets are known (Y, Z) and finite Set-valued uncertainty W Draper,Sahai (UC Berkeley) Universal Anytime ConCom7 6 / 23

Model of Channel Uncertainty: Compound Channels Channel W is known to be memoryless across time Input and output alphabets are known (Y, Z) and finite Set-valued uncertainty W Nature can choose any particular W W Choice remains fixed for all time Draper,Sahai (UC Berkeley) Universal Anytime ConCom7 6 / 23

Model of Channel Uncertainty: Compound Channels Channel W is known to be memoryless across time Input and output alphabets are known (Y, Z) and finite Set-valued uncertainty W Nature can choose any particular W W Choice remains fixed for all time Capacity well understood: C = sup Q inf W W I(Q, W) Draper,Sahai (UC Berkeley) Universal Anytime ConCom7 6 / 23

Review: Entirely noiseless channel t Window known to contain X t λ t will grow by factor of λ > Encode which control U t to apply Sending R bits, cut window by a factor of 2 R grows by Ω on each side 2 giving a new window for X t+ t+ As long as R > log 2 λ, we can have stay bounded forever Draper,Sahai (UC Berkeley) Universal Anytime ConCom7 8 / 23

Review: Delay-universal (anytime) communication B B 2 B 3 B 4 B 5 B 6 B 7 B 8 B 9 B B B 2 B 3 Y Y 2 Y 3 Y 4 Y 5 Y 6 Y 7 Y 8 Y 9 Y Y Y 2 Y 3 Y 4 Y 5 Y 6 Y 7 Y 8 Y 9 Y 2 Y 2 Y 22 Y 23 Y 24 Y 25 Y 26 Z Z 2 Z 3 Z 4 Z 5 Z 6 Z 7 Z 8 Z 9 Z Z Z 2 Z 3 Z 4 Z 5 Z 6 Z 7 Z 8 Z 9 Z 2 Z 2 Z 22 Z 23 Z 24 Z 25 Z 26 bb bb 2 bb 3 bb 4 bb 5 bb 6 bb 7 bb 8 bb 9 fixed delay d = 7 Draper,Sahai (UC Berkeley) Universal Anytime ConCom7 / 23

Review: Delay-universal (anytime) communication B B 2 B 3 B 4 B 5 B 6 B 7 B 8 B 9 B B B 2 B 3 Y Y 2 Y 3 Y 4 Y 5 Y 6 Y 7 Y 8 Y 9 Y Y Y 2 Y 3 Y 4 Y 5 Y 6 Y 7 Y 8 Y 9 Y 2 Y 2 Y 22 Y 23 Y 24 Y 25 Y 26 Z Z 2 Z 3 Z 4 Z 5 Z 6 Z 7 Z 8 Z 9 Z Z Z 2 Z 3 Z 4 Z 5 Z 6 Z 7 Z 8 Z 9 Z 2 Z 2 Z 22 Z 23 Z 24 Z 25 Z 26 bb bb 2 bb 3 bb 4 bb 5 bb 6 bb 7 bb 8 bb 9 fixed delay d = 7 Fixed-delay reliability α is achievable if there exists a sequence of encoder/decoder pairs with increasing end-to-end delays d j such that lim j d j ln P(B i ˆB j i ) = α Draper,Sahai (UC Berkeley) Universal Anytime ConCom7 / 23

Review: Delay-universal (anytime) communication B B 2 B 3 B 4 B 5 B 6 B 7 B 8 B 9 B B B 2 B 3 Y Y 2 Y 3 Y 4 Y 5 Y 6 Y 7 Y 8 Y 9 Y Y Y 2 Y 3 Y 4 Y 5 Y 6 Y 7 Y 8 Y 9 Y 2 Y 2 Y 22 Y 23 Y 24 Y 25 Y 26 Z Z 2 Z 3 Z 4 Z 5 Z 6 Z 7 Z 8 Z 9 Z Z Z 2 Z 3 Z 4 Z 5 Z 6 Z 7 Z 8 Z 9 Z 2 Z 2 Z 22 Z 23 Z 24 Z 25 Z 26 bb bb 2 bb 3 bb 4 bb 5 bb 6 bb 7 bb 8 bb 9 fixed delay d = 7 Reliability α is achievable delay-universally or in an anytime fashion if a single encoder works for all sufficiently large delays d Draper,Sahai (UC Berkeley) Universal Anytime ConCom7 / 23

Review: Delay-universal (anytime) communication B B 2 B 3 B 4 B 5 B 6 B 7 B 8 B 9 B B B 2 B 3 Y Y 2 Y 3 Y 4 Y 5 Y 6 Y 7 Y 8 Y 9 Y Y Y 2 Y 3 Y 4 Y 5 Y 6 Y 7 Y 8 Y 9 Y 2 Y 2 Y 22 Y 23 Y 24 Y 25 Y 26 Z Z 2 Z 3 Z 4 Z 5 Z 6 Z 7 Z 8 Z 9 Z Z Z 2 Z 3 Z 4 Z 5 Z 6 Z 7 Z 8 Z 9 Z 2 Z 2 Z 22 Z 23 Z 24 Z 25 Z 26 bb bb 2 bb 3 bb 4 bb 5 bb 6 bb 7 bb 8 bb 9 fixed delay d = 7 The anytime capacity Cany(α) is the supremal rate at which reliability α is achievable in a delay-universal way Draper,Sahai (UC Berkeley) Universal Anytime ConCom7 / 23

Review: Separation theorem for scalar control Necessity: If a scalar system with parameter λ > can be stabilized with finite η-moment across a noisy channel, then the channel with noiseless feedback must have Cany(η ln λ) ln λ In general: If P( X > m) < f(m), then K : Perror(d) < f(kλ d ) Sufficiency: If there is an α > η ln λ for which the channel with noiseless feedback has Cany(α) > ln λ then the scalar system with parameter λ with a bounded disturbance can be stabilized across the noisy channel with finite η-moment Draper,Sahai (UC Berkeley) Universal Anytime ConCom7 / 23

Review: Separation theorem for scalar control Necessity: If a scalar system with parameter λ > can be stabilized with finite η-moment across a noisy channel, then the channel with noiseless feedback must have Cany(η ln λ) ln λ In general: If P( X > m) < f(m), then K : Perror(d) < f(kλ d ) Sufficiency: If there is an α > η ln λ for which the channel with noiseless feedback has Cany(α) > ln λ then the scalar system with parameter λ with a bounded disturbance can be stabilized across the noisy channel with finite η-moment Proved using a direct equivalence so it also holds for compound channels Draper,Sahai (UC Berkeley) Universal Anytime ConCom7 / 23

From Rate/Reliability to Gain/Moments Error Exponent (base e) 2 8 6 4 2 2 3 4 5 6 Rate (in nats) Draper,Sahai (UC Berkeley) Universal Anytime ConCom7 2 / 23

From Rate/Reliability to Gain/Moments 8 Moments stabilized 6 4 2 2 4 6 8 Open loop unstable gain Draper,Sahai (UC Berkeley) Universal Anytime ConCom7 2 / 23

Why the random-coding bound works: tree codes 2 3 4 5 6 7 8 Time ML path decoding Log likelihoods add along path Disjoint segments are pairwise independent of the true path Er (R) analysis applies at the suffix Tree with iid random labels: Data chooses a path through the tree Transmit the path labels through the channel Feedback is not used Draper,Sahai (UC Berkeley) Universal Anytime ConCom7 4 / 23

Why the random-coding bound works: tree codes 2 3 4 5 6 7 8 Time Tree with iid random labels: Data chooses a path through the tree Transmit the path labels through the channel Feedback is not used ML path decoding Log likelihoods add along path Disjoint segments are pairwise independent of the true path Er (R) analysis applies at the suffix Shortest suffix dominates Achieves P e (d) K exp( E r (R)d) for every d for all R < C Draper,Sahai (UC Berkeley) Universal Anytime ConCom7 4 / 23

Relevant aspects of block and anytime codes Block coding: source bits, b group into blocks encode blocks independently y channel W(z y) z decode blocks independently Rare blocks in error Erroneous blocks never recovered Eventually results in exponential instability Anytime coding: source bits, b causally encode w/ tree code y = f(b, b, b ) k 2 k channel W(z y) z streaming decoder Can revisit earlier bit estimates Estimate reliabilities increase with delay Eventually detect and compensate for any incorrect controls Draper,Sahai (UC Berkeley) Universal Anytime ConCom7 5 / 23

Outline Problem setup Channel uncertainty and stabilization Review of past results without uncertainty 2 New result: universal anytime codes 3 Sufficient condition for stabilization 4 Conclusion Draper,Sahai (UC Berkeley) Universal Anytime ConCom7 6 / 23

Review: universal block codes for compound channels A single input distribution Q must be chosen for all W W Look at the empirical mutual information (EMI) between Z and candidate codewords Y m Choose the one with the highest mutual information Why does this work? The true codeword gives rise to an empirical channel that is like the true channel There are only a polynomial (n + ) Y Z set of joint types The random-coding error exponent Er (R, Q, W) is achieved Draper,Sahai (UC Berkeley) Universal Anytime ConCom7 7 / 23

Review: universal block codes for compound channels A single input distribution Q must be chosen for all W W Look at the empirical mutual information (EMI) between Z and candidate codewords Y m Choose the one with the highest mutual information Why does this work? The true codeword gives rise to an empirical channel that is like the true channel There are only a polynomial (n + ) Y Z set of joint types The random-coding error exponent Er (R, Q, W) is achieved What are the difficulties in generalizing to trees? Draper,Sahai (UC Berkeley) Universal Anytime ConCom7 7 / 23

Review: universal block codes for compound channels A single input distribution Q must be chosen for all W W Look at the empirical mutual information (EMI) between Z and candidate codewords Y m Choose the one with the highest mutual information Why does this work? The true codeword gives rise to an empirical channel that is like the true channel There are only a polynomial (n + ) Y Z set of joint types The random-coding error exponent Er (R, Q, W) is achieved What are the difficulties in generalizing to trees? The empirical mutual information (EMI) is not additive Ex: (,) + (,) gives 4 + 4 Draper,Sahai (UC Berkeley) Universal Anytime ConCom7 7 / 23

Review: universal block codes for compound channels A single input distribution Q must be chosen for all W W Look at the empirical mutual information (EMI) between Z and candidate codewords Y m Choose the one with the highest mutual information Why does this work? The true codeword gives rise to an empirical channel that is like the true channel There are only a polynomial (n + ) Y Z set of joint types The random-coding error exponent Er (R, Q, W) is achieved What are the difficulties in generalizing to trees? The empirical mutual information (EMI) is not additive Ex: (,) + (,) gives 4 + 4 The polynomial term grows with n not with delay d Delays must be longer as time goes on Draper,Sahai (UC Berkeley) Universal Anytime ConCom7 7 / 23

Sequential Suffix-EMI Decoding Decode sequentially, using max empirical mutual inform (EMI) comparisons at each stage If a blue codeword has the max EMI, decode first bit to, else to If a blue codeword suffix has the max EMI, decode second bit to, else to If a blue codeword suffix has the max EMI, decode third bit to, else to Draper,Sahai (UC Berkeley) Universal Anytime ConCom7 8 / 23

The universal anytime coding theorem Given a rate R > and a compound discrete memoryless channel W, there exists a random anytime code such that for all E < E any,univ (R) there is a constant K > such that Pr[ B n d B n d ] K2 de for all n, d where E any,univ (R) = sup inf inf D(P V Q W) + max{, I(P, V) R} Q W W P,V = sup inf E r(r, Q, W) Q W W No feedback is needed Does as well as could be hoped for essentially hits E r (R) Draper,Sahai (UC Berkeley) Universal Anytime ConCom7 9 / 23

Back to the stabilization problem Can use sup Q inf W W E r (R, Q, W) to evaluate compound channels W Draper,Sahai (UC Berkeley) Universal Anytime ConCom7 2 / 23

Back to the stabilization problem Can use sup Q inf W W E r (R, Q, W) to evaluate compound channels W Picking Q is unavoidable determines codebook and capacity Draper,Sahai (UC Berkeley) Universal Anytime ConCom7 2 / 23

Back to the stabilization problem Can use sup Q inf W W E r (R, Q, W) to evaluate compound channels W Picking Q is unavoidable determines codebook and capacity But evaluating this bound can be difficult Is there a simpler condition that only depends on capacity C(W)? Gallager Exercise 523 tells us: E r (R, Q, W) (I(Q, W) R) 2 /( 8 e 2 + 4(ln Z ) 2 ) Translate to η-stabilization: C(W) log λ > (2 log Z ) η log λ 2(log e)2 ( + log e e 2 (log Z ) 2 ) Says that O( η log λ) extra capacity suffices to get η-th moment stabilized Draper,Sahai (UC Berkeley) Universal Anytime ConCom7 2 / 23

Visualizing the new universal sufficient condition Error Exponent (base e) 2 8 6 4 2 2 3 4 5 6 Rate (in nats) Draper,Sahai (UC Berkeley) Universal Anytime ConCom7 22 / 23

Visualizing the new universal sufficient condition 8 Moments stabilized 6 4 2 2 4 6 8 Open loop unstable gain Draper,Sahai (UC Berkeley) Universal Anytime ConCom7 22 / 23

Conclusion Random anytime codes exist for compound channels Can thus stabilize systems over uncertain channels Draper,Sahai (UC Berkeley) Universal Anytime ConCom7 23 / 23

Conclusion Random anytime codes exist for compound channels Can thus stabilize systems over uncertain channels Can operate with coarse description of channel uncertainty Channel input distribution Q Resulting capacity C Set-size proxy: output alphabet size Z Draper,Sahai (UC Berkeley) Universal Anytime ConCom7 23 / 23

Conclusion Random anytime codes exist for compound channels Can thus stabilize systems over uncertain channels Can operate with coarse description of channel uncertainty Channel input distribution Q Resulting capacity C Set-size proxy: output alphabet size Z Performance loss for coarse channel uncertainty Bounds should be improved Draper,Sahai (UC Berkeley) Universal Anytime ConCom7 23 / 23

Conclusion Random anytime codes exist for compound channels Can thus stabilize systems over uncertain channels Can operate with coarse description of channel uncertainty Channel input distribution Q Resulting capacity C Set-size proxy: output alphabet size Z Performance loss for coarse channel uncertainty Bounds should be improved Feedback should be used Draper,Sahai (UC Berkeley) Universal Anytime ConCom7 23 / 23