FEEDBACK does not increase the capacity of a discrete

1 Sequential Differential Optimization of Incremental Redundancy Transmission Lengths: An Example with Tail-Biting Convolutional Codes Nathan Wong, Kasra Vailinia, Haobo Wang, Sudarsan V. S. Ranganathan, and Richard D. Wesel Email: {nsc.wong, vailinia, whb12, sudarsanvsr, wesel}@ucla.edu Abstract This paper applies the sequential differential optimization SDO algorithm to optimize the transmission lengths of incremental redundancy for a 124-state tail-biting convolutional code. The tail-biting reliability-output Viterbi algorithm is used to determine whether to inform the transmitter that a message has been successfully received or to request that the transmitter provide additional convolutional code bits. In order to maximize the average throughput, SDO is used to determine the rate of the initial codeword and the number of bits of incremental redundancy to be sent in each increment. With the help of SDO, this paper demonstrates a system that achieves 86.3 percent of the binary-input AWGN capacity for SNR 2 db with an average bloclength of 115.5 symbols. I. INTRODUCTION FEEDB does not increase the capacity of a discrete memoryless channel [1], but it does provide benefits. Polyansiy et al. [2] show that transmitting incremental redundancy in response to feedbac can approach capacity at significantly shorter average bloclengths than a system without feedbac. This paper continues an exploration of incremental redundancy with feedbac where the transmission lengths have been optimized using sequential differential optimization SDO. The paper focuses on the specific example of a 124- state tail-biting convolutional code [3] on the 2 db binary-input BI additive white Gaussian noise AWGN channel. Short pacet communications are important for future wireless systems [4], and Liva et al. [5] confirm that tail-biting convolutional codes provide competitive performance at the short bloclengths considered in this paper, maing the use of TBCCs interesting in this context. For a message of bits of information, we consider a system that, based on feedbac, sends at most m transmissions that have lengths in symbols l 1,..., l m. To prove theoretical limits, Polyansiy et al. [2] focus on the limiting case in which feedbac is used to decide whether or not to transmit each subsequent symbol of possible incremental redundancy. Thus, the analysis of [2] considers a possibly infinite number of increments of length 1, i.e. m = and l j = 1, j. Williamson et al. [6] began to explore how closely capacity might be approached practically by using feedbac on the This research is supported in part by NSF grants CCF-116251 and CCF-1618272. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the NSF. This wor used computational and storage services associated with the Hoffman2 Shared Cluster provided by UCLA Institute for Digital Research and Education s Research Technology Group. BI-AWGN channel. Focusing on average bloclengths less than 15 symbols, they use a tail-biting convolutional code TBCC with a hybrid ARQ that uses a small number m = 5 of incremental transmissions. Incremental transmissions are facilitated by pseudo-random rate-compatible puncturing [7] [8] of a 124-state, rate-1/3 convolutional mother code with polynomials 2325, 2731, 3747 in octal. In [6], after receiving the j th increment, the tail-biting reliability-output Viterbi algorithm TB-ROVA developed in [9] provides the decoder with reliability information. The receiver uses this reliability information to instruct the transmitter via feedbac either to terminate or to send additional incremental redundancy in the j + 1 th transmission. Williamson et al. show simulation results in [6] that exceed the random-coding lower bound on achievability given in Theorem 3 of [2] using m = 5, with transmission lengths l 1,..., l 5 optimized by an exhaustive search. The complexity of optimization by exhaustive search prevented exploration of performance beyond m = 5 in [6]. Vailinia et al. [1] explore approaching capacity for larger average bloclengths approximately 15-5 symbols using a rate-compatible system based on non-binary LDPC codes with incremental redundancy, where feedbac requests additional incremental transmissions when the cyclic redundancy chec does not pass. Using m = 1 possible incremental transmissions, Vailinia et al. achieve a throughput of 93% of capacity at a frame error rate of 1 3 for an average bloclength of about 5 symbols. This result was facilitated by the technique of sequential differential optimization SDO developed in [1] that enables optimization of the lengths l 1,..., l m of numerous incremental transmissions without the need for exhaustive search. Empowered by the newly available SDO technique of [1], this paper returns to the question of what can be achieved with the system initially studied in [6]: a hybrid ARQ with average bloclengths below 15 symbols and using a 124-state tailbiting convolutional code and the TB-ROVA of [9] to decide when to request additional transmissions. The rest of the paper proceeds as follows: Section II provides a description of SDO. Section III shows the improved throughput achievable with increased m as well as a calculation of the throughput possible for this system in the limiting case when l j = 1 essentially m =. Section IV shows that for small enough values, the Gaussian rate model becomes inaccurate, and Section V concludes the paper.

2 II. SEQUENTIAL DIFFERENTIAL OPTIMIZATION SDO Consider a system that communicates a -bit message by using incremental redundancy to send up to m possible transmissions in an accumulation cycle. The transmissions have lengths of l 1,..., l m, where sending each additional transmission depends on /N feedbac. Each subsequent attempt in the accumulation cycle has the advantage of a successively larger cumulative bloclength of N i, where i N i = l j. 1 j=1 The decision of whether to send an to terminate the transmission or to send a N to request additional redundancy is based on an indicator of reliable decoding that is available at the receiver. In this paper that indicator is a reliability function computed by the ROVA algorithm of [9]. The receiver does not now whether it has truly decoded correctly; the ROVA threshold is designed to achieve a desired target probability of undetected error P UE. A decoding error is only possible when the receiver maes a final decision and an is sent, and all such errors are undetected. Let P Nj Nj and P N be the marginal probabilities of a decoding success or failure based on the reliability indicator when the decoder is presented with a received codeword having bloclength N j. Note that P Nj + P Nj N = 1. If decoding is still unsuccessful after all m decoding attempts in the accumulation cycle, the associated -bit message is not lost, rather the transmission is attempted again from scratch. This is referred to by Heindlmaier and Soljanin in [11] as a fixed incremental redundancy scheme, but as shown in [11] the loss from the infinite incremental redundancy scheme where m = is small when the failure rate is low. Let I be the number of successfully transmitted information bits in an accumulation cycle. Let N be the number of symbols transmitted in an accumulation cycle. The throughput rate R t is defined as where E[N] N 1 P N1 R t = E[I] E[N] = + m j=2 1 P Nm N P UE N j [ P Nj E[N], 2 ] P Nj 1 + N m P Nm N. The expression for E[N] above is an approximation because it assumes that if an was sent when decoding a message of length N j 1, then certainly an would also be sent when decoding the corresponding longer message with length N j. While observed to be true for the non-binary LDPC codes explored in [1], this is not true in general and specifically not the case for the convolutional codes explored in this paper. However, events where an would be followed by a N are relatively rare, and ignoring this effect simply leads to a slight underestimate of throughput by under-counting the s that occur for the first time on the j th attempt. To further simplify the optimization of R t, noting that E[I], maximizing R t is essentially equivalent to minimizing E[N]. TABLE I GAUSSIAN APPROXIMATION RESULTS OBTAINED USING BASED ON 1, AND 5, SAMPLES TO CREATE HISTOGRAM MODELED AS A GAUSSIAN CURVE TO FIND MEAN AND VARIANCE TO BE USED WITH SDO FOR m = 2 DECODING ATTEMPTS AND = 64 INFORMATION BITS Number of Samples Category 1 5 Mean 672 666 Standard Deviation.572.573 Transmission Lengths l j 2, 2, 2, 2, 2, 3, 3, 2, 2, 2, 2, 2, 3, 3, 96, 4, 3, 2, 2, 2, 2, 96, 4, 3, 2, 2, 2, 2, 3, 4, 5, 6, 1, 21 3, 4, 5, 6, 9, 18 R t 5447 51536 λ 115.5146 115.9986 At each bloclength N j there is an associated rate /N j at which decoding is either successful or not. In [1], the probability of the rate R s being the first successful decoding rate is modeled as a Gaussian distribution. Section II-A shows that this model is sufficiently accurate for a message size of = 64 for the specific system in question. Section II-B describes how SDO sequentially sets relevant derivatives to zero to identify an optimal set of bloclengths N 1,..., N m that minimizes E[N]. A. A Gaussian Approximation for the Highest Successful Rate To empirically determine a model for P Nj, or equivalently R s the decoder successively attempts decoding with the bloclength increasing one symbol at a time until the message is decoded successfully. This is performed for multiple iterations until an empirical p.m.f. is obtained with sufficient resolution and confidence. This must be done separately for each message length of bits that is considered. To model the empirical p.m.f. with a Gaussian, the mean and variance are estimated using linear regression applied to the following model P Nj Q /Nj µ s, 3 Here, Q is the probability of the tail of a Gaussian with zero mean and unit variance. The variable R s = /N j represents the rate at which a codeword is first successfully decoded. We can rearrange 3 to get the following: µ + Q 1 P Nj = R s, 4 where µ and are regression coefficients that we estimate by linear regression techniques to solve for mean and variance. We found that 1, simulation points are sufficient to estimate the regression coefficients for the = 64 case. A larger number of simulation points did not significantly affect the accuracy of the mean and variance estimates. Table I compares the linear regression estimates of mean and variance values for the example case of = 64 and m = 2 with the number of simulated points being 1, and 5, to illustrate that 1, is sufficient. Table I also provides the resulting values of l 1,..., l 2, the average bloclength λ = E[N], and the rate R t for these two examples.

3 Decoding Probability 1.8 Probability of Acnowledgement for = 64 GR Approximation 8 1 12 14 16 18 Bloclength N i Fig. 1. Empirical probability of acnowledgement P N j as blac asteriss as a function of the bloclength N j for 5, simulation points for decoding the variable-rate 124-state TBCC over the AWGN channel with SNR 2. db and = 64. Also shown is the Gaussian approximation using the mean and variance computed by linear regression. Fig. 1 shows, for the = 64 case, the empirical probability of acnowledgement P Nj as blac asteriss as a function of the bloclength N j for 5, simulation points and also plots the approximation using the Gaussian rate model with the mean and variance computed by linear regression. We see that the Gaussian approximation fits the curve well. B. A Tree-based Approach for SDO Let, µ, and be fixed. For a given value of N 1, the optimal values of N i for i > 1 can be determined by solving a set of differential equations. Sequentially setting the derivatives of E[N] with respect to N i to zero as described in [1] yields the following equations: Q N 2 = N i = N s 1 Q N i 1 + N 1 Q N s 1 5 Q N s 1 + N i 1 Q Q N i 1 N i 1 Q N i 2 where 6 is for the case where i > 2. While 5 and 6 yield real numbers, the actual N i values must be integers. We create a tree that stores both the floor and ceiling of each solution, using 6 for both values until all m values of N i have been computed. Thus 2 m 1 sequences of potential bloclengths N 1,..., N m are computed for each candidate value of N 1 that is considered. Finally, from among these possibilities, the sequence of bloclengths N 1,..., N m that meets the required frame error rate FER and minimizes E[N] is selected. A lower complexity alternative is simply to compute all values of N i as real numbers. As a final step, each of these N i 6 TABLE II GAUSSIAN APPROXIMATION RESULTS FOR FLOOR-AND-CEILING AND STANDARD-ROUNDING APPROACHES OF CALCULATING TRANSMISSION LENGTHS USING SDO FOR m = 16 AND = 48 INFORMATION BITS SDO Approach and Method Category floor and ceiling standard rounding Transmission Lengths 73, 3, 2, 2, 2, 74, 3, 2, 2, 2, 2, 2, 2, 2, 2, 3, 2, 2, 2, 2, 2, 2, 3, 4, 5, 7, 14 2, 2, 2, 2, 2 Rate 48513 33947 values is rounded in the standard way. This produces a single sequence of potential bloclengths N 1,..., N m rather than the 2 m 1 sequences produced by the floor-and-ceiling approach. Table II compares the tree-based floor-and-ceiling approach to the standard-rounding approach for = 48 and m = 16. Not surprisingly, the simpler standard-rounding approach achieves lower rates than the floor-and-ceiling approach. The benefits of the floor-and-ceiling approach go beyond an exhaustive solution to the integer constraint problem. The SDO equations 5 and 6 inherently assume an infinite number of transmissions can be sent, but we will only use the first m. The tree produced by the floor-and-ceiling provides a set of solutions in the neighborhood of the infinite transmissions solutions from which the best solution for a finite m value can be identified. This is seen in Table II where the floorand-ceiling approach uses longer increments at the end of the finite-length horizon whereas the standard approach does not. For large values of m, it is computationally prohibitive to perform a full floor-and-ceiling search for every possible value of N 1. For these cases, we used a "targeted" floor-and-ceiling approach in which the SDO equations are used to produce a single real-valued N i sequence for each possible integervalued N 1. The N 1 whose induced real-valued sequence produced the best smallest value of E[N] is then used in a floor-and-ceiling search to identify the best set of integer N i values for that N 1. We used the full tree-based floor-andceiling approach below for m {4, 8, 16} and the targeted floor-and-ceiling approach for m = 32. III. APPLYING SDO TO 124-STATE TBCC WITH = 64 As an initial sanity chec, we compare the throughput results for m = 5 obtained using SDO to the results of [6] using exhaustive search. Table III confirms that the transmission lengths l 1,..., l 5 are very similar. Fig. 2 confirms that the throughput values are quite similar. TABLE III ES AND SDO TRANSMISSION LENGTHS FOR m = 5 FOR VARIOUS Transmission Lengths l 1,..., l 5 Info. Bits Exhaustive Search Sequen. Diff. Opt. 16 29, 4, 4, 4, 7 29, 3, 3, 4, 6 32 56, 5, 5, 7, 12 55, 5, 5, 7, 14 48 8, 7, 7, 9, 16 8, 6, 6, 8, 14 64 16, 9, 9, 12, 22 16, 8, 8, 1, 17 91 151, 13, 14, 17, 31 153, 12, 12, 16, 3

4.7 5 5 5 5 5 =16 =32 =48 =64 =91 BI-AWGN capacity Random coding lower bound [2] m= m=32, SDO m=5, Exhaustive Search [3] m=5, SDO R t =/E[N] =16,32,48,64,91 2 4 6 8 1 12 14 16 18 Expected Bloclength E[N] Fig. 2. Throughput rate R t as a function of average bloclength E[N] for the = 64, 124-state TBCC transmission scheme with lengths determined by SDO for m = 5 and m = 32 and by exhaustive search for m = 5 simulated over the BI-AWGN channel with SNR 2. db and target probability of error P UE = 1 3. Also shown for reference are the BI-AWGN capacity of 42 bits at 2 db, the random coding lower bound of [2], and the m = R t predicted by the Gaussian approximation illustrated in Fig. 1 A. Throughput with an Increased Number of Transmissions Fig. 2 shows throughput rate R t as a function of expected bloclength E[N] for the incremental redundancy system studied in Williamson et al. [6] using a 124-state TBCC with feedbac controlling m = 5 possible incremental transmissions. Larger values of m were not considered in [6] because exhaustive search could not find optimal transmission lengths for m > 5. However, even with m = 5 the system exceeds the random coding lower bound of [2] for 37 < E[N] < 8. SDO facilitates characterization of performance for larger values of m. Fig. 2 shows that a consistent throughput increase of about.26 bits is achieved by increasing m = 5 to m = 32 possible incremental transmissions. With m = 32, the random coding lower bound is exceeded for 35 < E[N] < 12, a larger range than for m = 5. Moreover, the theoretical curve in Fig. 2 for m = obtained by using the Gaussian approximation on R s shown in Fig. 1 reveals that no significant improvement is expected for m > 32. Fig. 2 shows that for = 91 the throughput R t decreases in contrast to the random coding lower bound, which monotonically increases with average bloclength. As pointed out in [6], convolutional code frame error rate performance degrades once the bloclength is beyond twice Anderson s analytical decision depth [12]. Thus, it is not surprising to see R t begin to decrease as bloclength increases for a convolutional code with a fixed number of memory elements. The highest throughput point is = 64 and m = 32 which achieves 86.3% of BI-AWGN capacity with an E[N] of 115.5 symbols as compared to the m = 5 system which achieves 82.2% of capacity with an E[N] of 121.2 symbols. Receiver complexity and overall system complexity increase with m. However, as shown in Figs. 3 and 4, the complexity increase is less than might be expected. Fig. 3 shows that the average number of transmissions for the = 64 case that Average Number of Transmissions 25 2 15 1 5 4 8 16 32 11 Maximum Number of Transmission Lengths m Fig. 3. Average number of transmissions in an accumulation cycle as a function of m for = 64 TBCC with lengths determined by SDO except for m = 11 where all lengths are 1 simulated over the BI-AWGN channel with SNR 2. db and target probability of error P UE = 1 3. Average / m 4 8 16 32 11 Maximum Number of Transmission Lengths m Fig. 4. Ratio of the average number of transmissions in an accumulation cycle divided by m for the same system and channel as Fig. 3 maximizes throughput is 1 even when m = 32. Fig. 4 shows that as m increases the average number of transmissions becomes a smaller fraction of m. Indeed, when m = 11, which is essentially m = since the increments are all a single symbol and decoding starts at N 1 = 92 which has a very low P 92 from Fig. 1, the average number of transmissions for the = 64 system computed by simulation is only 23.6. Fig. 5 shows simulated throughput for m {4, 8, 16, 32}. Throughput increases with m, but by m = 32, throughput results are near the limiting curve of m =. 6 4 2 8 6 4 =16 =32 =48 =64 =91 m=4 m=8 m=16 m=32 m= R t =/E[N] =16,32,48,64,91 2 2 4 6 8 1 12 14 16 18 2 Expected Bloclength E[N] Fig. 5. R t as a function of E[N] for the same channel as Fig. 2 with lengths determined by SDO for various values of m.

5 6 4 2 8 6 4 2 = 64 115 E[N] 125 = 32 61 E[N] 66 = 16 34 E[N] 38 = 91 166 E[N] 177 GR-Model 4 8 16 32 Number of Transmissions log 2 m Probability of Acnowledgement 1.9.8.7.1 Probability of Acnowledgement for = 16 GR Approximation 2 25 3 35 4 45 Bloclength N j Fig. 6. R t as a function of m for various values of both from simulation and computed using the Gaussian rate model shown in Fig. 1 for the 124- state TBCC over the BI-AWGN channel with SNR 2. db. Fig. 7. Empirical probability of acnowledgement and its approximation based on a Gaussian rate model for decoding the variable-rate 124-state TBCC over the AWGN channel with SNR 2. db and = 16. IV. GAUSSIAN RATE MODEL INACCURATE FOR < 64 The Gaussian rate model for computing P Nj in 3 can be used to compute an approximation of R t using 2, which can be compared with the simulated R t. Fig. 6 compares R t results obtained by simulation with R t values computed using the Gaussian model illustrated in Fig. 1. For lower values of, the difference between the values computed using the model and the simulation results is significant. This reveals limitations in the Gaussian model of rate for smaller values of. To further examine the limitations of the Gaussian rate model, Fig. 7 shows, for the = 16 case, the empirical probability of acnowledgement P Nj as blac asteriss as a function of the bloclength N j for 5, simulation points and also plots the approximation using the Gaussian rate model with the mean and variance computed by linear regression. For this case, the Gaussian rate model simply does not match the behavior produced by simulation. In a future wor, we explore this disparity and develop a more refined model that wors well even for these very small values of. V. CONCLUSION Sequential differential optimization SDO, introduced in [1], provides a way to optimize a large number of incremental transmission lengths to maximize the throughput of systems that use /N feedbac to control transmissions of incremental redundancy. In this paper, we used SDO to improve the throughputs obtained in [6] by using a larger number of incremental transmissions. As in [1], for the = 64 case, the Gaussian rate model provides an accurate model of the probability of successful decoding and acnowledgement observed in simulation. For the = 64 case, by increasing the maximum number of incremental transmissions from m = 5 to m = 32, the throughput R t was increased from 82.2% of BI-AWGN capacity to 86.3%, a value which exceeds the random coding lower bound of [2] at its corresponding average bloclength of 115.5 symbols. Although the Gaussian rate model proved to be excellent for = 64, for smaller values of it does not accurately characterize the probability of acnowledgement. An area of ongoing investigation is to develop a model that remains accurate for small values of. Another area of investigation is to explicitly include the finite-m constraint in the SDO computation. As it is, SDO optimizes transmission lengths assuming an infinite sequence of transmissions. Finally, TBCCs with more states should achieve an even greater percentage of capacity. REFERENCES [1] C. E. Shannon, The zero error capacity of a noisy channel, IRE Trans. Inf. Theory, vol. 2, no. 3, pp. 8 19, Sep. 1956. [2] Y. Polyansiy, H. V. Poor, and S. Verdu, Feedbac in the nonasymptotic regime, IEEE Trans. Inf. Theory, vol. 57, no. 8, pp. 493 4925, August 211. [3] H. Ma and J. Wolf, On tail biting convolutional codes, IEEE Trans. Comm., vol. 34, no. 2, pp. 14 111, Feb 1986. [4] G. Durisi, T. Koch, and P. Popovsi, Toward massive, ultrareliable, and low-latency wireless communication with short pacets, Proc. of the IEEE, vol. 14, no. 9, pp. 1711 1726, Sept 216. [5] G. Liva, L. Gaudio, N. Tudor, and T. Jerovits, Code design for short blocs: A survey, arxiv, 216, 161.873v1. [6] A. R. Williamson, T.-Y. Chen, and R. D. Wesel, Variable-length convolutional coding for short bloclengths with decision feedbac, IEEE Trans. Comm., vol. 63, no. 7, pp. 2389 243, July 215. [7] J. Hagenauer, Rate-compatible punctured convolutional codes rcpc codes and their applications, IEEE Trans. Comm., vol. 36, no. 4, pp. 389 4, Apr 1988. [8] K. M. Rege and S. Nanda, Irreducible FER for convolutional codes with random bit puncturing: application to cdma forward channel, in Proc. of Veh.Tech. Conf. - VTC, vol. 2, Apr 1996, pp. 1336 134 vol.2. [9] A. R. Williamson, M. J. Marshall, and R. D. Wesel, Reliabilityoutput decoding of tail-biting convolutional codes, IEEE Trans. Comm., vol. 62, no. 6, pp. 1768 1778, June 214. [1] K. Vailinia, S. V. S. Ranganathan, D. Divsalar, and R. D. Wesel, Optimizing transmission lengths for limited feedbac with nonbinary ldpc examples, IEEE Trans. Comm., vol. 64, no. 6, pp. 2245 2257, June 216. [11] M. Heindlmaier and E. Soljanin, Isn t hybrid ARQ sufficient? in 52nd Annu. Allerton Conf. Commun., Control, and Computing Allerton, Sep. 214, pp. 563 568. [12] J. B. Anderson and K. Balachandran, Decision depths of convolutional codes, IEEE Trans. Inf. Theory, vol. 35, no. 2, pp. 455 459, Mar 1989.