Error-Correction Coding for Digital Communications

Error-Correction Coding for Digital Communications

Applications of Communications Theory Series Editor: R. W. Lucky, Bell Laboratories INTRODUCTION TO COMMUNICATION SCIENCE AND SYSTEMS John R. Pierce and Edward C. Posner OPTICAL FIBER TRANSMISSION SYSTEMS Stewart D. Personick TELECOMMUNICATIONS SWITCHING J. Gordon Pearce ERROR-CORRECTION CODING FOR DIGITAL COMMUNICATIONS George C. Clark, Jr., and J. Bibb Cain A Continuation Order Plan is available for this series. A continuation order will bring delivery of each new volume immediately upon publication. Volumes are billed only upon actual shipment. For further information please contact the publisher.

Error-Correction Coding for Digital Communications George C. Clark, Jr. and J. Bibb Cain Harris Corporation Melbourne, Florida SPRINGER SCIENCE+BUSINESS MEDIA, LLC

Library of Congress Cataloging in Publication Data Oark, George C. (George Cyril), 1938- Error-correction coding for digital communications. Bibliography: p. Includes index. 1. Data transmission systems. 2. Error-correcting codes (Information theory). I. Cain, J. Bibb. II. Title. TK5102.5.C52 621.38'0413 ISBN 978-1-4899-2176-5 ISBN 978-1-4899-2174-1 (ebook) DOI 10.1007/978-1-4899-2174-1 81-1630 AACR2 15 14 13 12 11 10 9 8 1981 Springer Science+Business Media New York Originally published by Plenum Press, New York in 1981 Softcover reprint of the hardcover 1st edition 1981 All rights reserved No part of this book may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, microfilming, recording, or otherwise, without written permission from the Publisher

Preface Error-correction coding is being used on an almost routine basis in most new communication systems. Not only is coding equipment being used to increase the energy efficiency of communication links, but coding ideas are also providing innovative solutions to many related communication problems. Among these are the elimination of intersymbol interference caused by filtering and multipath and the improved demodulation of certain frequency modulated signals by taking advantage of the "natural" coding provided by a continuous phase. Although several books and numerous articles have been written on coding theory, there are still noticeable deficiencies. First, the practical aspects of translating a specific decoding algorithm into actual hardware have been largely ignored. The information that is available is sketchy and is widely dispersed. Second, the information required to evaluate a particular technique under situations that are encountered in practice is available for the most part only in private company reports. This book is aimed at correcting both of these problems. It is written for the design engineer who must build the coding and decoding equipment and for the communication system engineer who must incorporate this equipment into a system. It is also suitable as a senior-level or first-year graduate text for an introductory one-semester course in coding theory. The book U"Ses a minimum of mathematics and entirely avoids the classical theorem/proof approach that is often seen in coding texts. Whenever possible heuristic arguments are advanced and concepts are developed by drawing analogies. We have intentionally constrained the level of mathematical rigor used in presenting error-correction concepts. Notwithstanding, coding is an extremely mathematical subject. and it would be an impossible task to attempt to present the material with no mathematics at all. The assumption is made that the practicing engineer is mostly interv

vi Preface ested in using the mathematics and not in constructing rigorous proofs. Thus, the goal is to develop a manipulative and intuitive understanding through numerous examples. Although there are many perils in this approach, a capable engineer can usually verify that a particular procedure is correct by trying it on a few test cases. The serious student may later wish to round out his or her understanding by pursuing the subject to its very roots. In addition to the large number of papers in the open literature. the excellent texts by Peterson and Weldon,< 'l Berlekamp, <ll and Gallager. <Jl are highly recommended for this purpose. Not all aspects of coding theory are treated in equal depth. Some topics, such as coding bounds, are treated very lightly while some others, such as certain classes of rarely used codes, are omitted entirely. Numerous classes of codes of practical interest are discussed in terms of methods of construction and code properties. However, there exist additional interesting properties of these codes as well as other code classes which we have chosen to omit for reasons of brevity. Instead, we have chosen to emphasize the formulation and implementation of decoding algorithms. The viewpoint, the selection of topics, and the degree of emphasis largely reflects the authors' personal preference and is based upon 15 years of practical experience in designing and evaluating coding equipment. However. we feel that the approach we have taken will also provide the reader with the insight and motivation to undertake some of the more difficult topics in other texts. In this book the application of error-correction coding is presented in the context of communication system design. Again, this reflects the authors' personal preferences. These techniques may also be used in other applications such as in computer memory systems. We have attempted to develop the tools necessary to enable the reader to evaluate such applications and to select an appropriate coding/decoding technique. An introduction to the use of error-correction coding in digital communication systems is presented in Chapter 1. Following this, the next four chapters are devoted to block codes and the associated decoding techniques. Fundamental concepts of linear codes (parity check matrix, syndrome, shift-register encoders, etc.) are developed in Chapter 2. and several useful classes of codes are presented. The simple hard decision decoding techniques for block codes are discussed in Chapter 3. These include Meggitt decoders, information-set decoders, and threshold decoders. Special attention is given to those features which result in minimum implementation complexity. Chapter 4 presents a collection of techniques for decoding block codes utilizing demodulator "soft" decisions. These include techniques developed by Massey, Hartmann and Rudolph. Chase, and Weldon. Various hardware realizations are presented, and performance/ complexity considerations are discussed. Algebraic techniques for multiple-

Preface vii error correction are developed in Chapter 5. The theory is developed using a transform approach. Numerous examples are used, and various hardware configurations are discussed. Convolutional coding techniques are presented in the next two chapters. Basic structural properties of convolutional codes and the Viterbi algorithm are developed in Chapter 6. Numerous performance curves are presented, and special attention is given to performance/ complexity tradeoffs. A similar treatment is provided in Chapter 7 for the table look-up, threshold, and sequential decoding techniques. Finally, in Chapter 8 we discuss a number of important system applications including concatenated codes, coding for the white Gaussian noise channel, interleaver structures, burst-noise channels, coding for spread spectrum systems, and coding for bandwidth-constrained channels. By including an extensive treatment of applications, we believe this book will be extremely useful to the communication system engineer. Numerous lists of good codes are provided in Appendices A and B. There are a number of acknowledgments which we should make. A book of this nature relies heavily on the pioneering work of the many researchers in this field. We have tried to adequately acknowledge this work in the references cited throughout the text. The authors are grateful for having had the opportunity to work for the Harris Corporation, Government Systems Group where they have been able to develop errorcorrection hardware and to apply coding techniques in a variety of communication systems. We would also like to thank C. J. Palermo and R. E. Totty for their encouragement during the writing of this book. Finally, we would like to acknowledge the technical contributions provided by our colleagues J. M. Geist, R. C. Davis, R. W. Boyd, and G. H. Thaker, the reviews provided by D. B. Bradley, A. M. Bush, and L. D. Rudolph, and the long hours of assistance in preparation of the manuscript provided by Barbara Snow and Sandra Wilson. Special thanks must also go to our wives Ann and Sissie for their encouragement and support during the past year. Harris Corporation George C. Clark, 1 r. J. Bibb Cain

Contents 1. Fundamental Concepts of Coding 0 1010 Basic 1020 Practical 1030 Performance 1.40 Coding 1.50 Remarks Principles 0 Constraints 0 102010 Data 1.2020 Encoder 1.2030 Modulator 102.40 Waveform 1.2050 Demodulator 1.2060 Decoder 1.2070 Discrete Source 0 0 0 Channel 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 8 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 10 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 II 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 12 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 14 Data Channel 0 Computations 0 Computation of Error Rate 0 Bound Computation of Error Rate 0 of Error Detection Systems 0 Decision Decoding 0 Effects 0 of Noncoherent Combining 0 Gain 0 Bounds 0 on Minimum Distance 0 1.301. Exact 1.3020 Union 1.3030 Performance 1.3.40 Soft 103050 Quantization 103060 Effect 1.3070 Coding 1.401. Bounds 1.4020 Random Problems 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 18 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 18 0 0 0 0 0 0 0 0 0 0 0 0 0 23 0 0 0 0 0 0 0 0 0 0 0 0 0 0 24 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 26 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 30 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 34 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 35 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 37 Coding Bound 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 38 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 41 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 46 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 47 2. Group Codes 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 49 2010 Generalized Parity Check Codes 0 Check Matrix 0 Distance Considerations 0 Codes 0 201010 Parity 201 2 Minimum 201030 Hamming 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 50 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 52 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 53 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 54 ix

X Contents 2.1.4. Generator Matrix................................. 56 2.1.5. Dual Codes...................................... 57 2.1.6. The Syndrome................................... 58 2.1.7. The Penny-Weighing Problem....................... 59 2.1.8. A Simple Iterated Code............................ 61 2.2. Polynomial Codes...................................... 62 2.2.1. Finite Field Arithmetic............................. 63 2.2.2. Generation of Polynomial Codes..................... 68 2.2.3. Cyclic Codes..................................... 72 2.2.4. Alternate Encoder for Cyclic Codes................... 73 2.2.5. Additional Properties of Polynomials and Galois Field Elements........................................ 75 2.2.6. Codes Specified by Roots........................... 78 2.2.7. Syndrome Polynomials............................. 79 2.2.8. Modifications of Codes............................ 84 2.3. Important Classes of Group Codes......................... 85 2.3.1. BCH Codes..................................... 86 2.3.2. Golay Codes.................................... 88 2.3.3. Maximal-Length Codes............................ 89 2.3.4. Reed-Muller Codes............................... 91 2.3.5. Quadratic Residue Codes........................... 92 2.3.6. Remarks........................................ 94 Problem~................................................. 94 3. Simple Nonalgebraic Decoding Techniques for Group Codes... 97 3.1. Meggitt Decoders...................................... 97 3.2. Information Set Decoding................................ 102 3.2.1. General Decoding Algorithm........................ 102 3.2.2. Information Set Decoding Using the H Matrix.......... 107 3.2.3. Predetermined Covering Sets........................ 109 3.2.4. Random Search Techniques......................... 119 3.3. Threshold Decoding.................................... 131 3.4. Remarks............................................. 137 Problem~................................................. 138 4. Soft Decision Decoding of Block Codes....................... 141 4.1. APP Threshold Decoding................................ 143 4.1.1. Derivation of the APP Decoding Rule................. 144 4.1.2. Computation of the Weighting Terms................. 146 4.1.3. Approximate Computation of wi............ 148 4.1.4. Implementation of the APP Decoder.................. 150 4.2. Optimum Symbol-by-Symbol Decoding..................... 154 4.2.1. Derivation of the Hartmann-Rudolph Algorithm........ 155 4.2.2. An Alternate Form of the HR Algorithm... 156 4.2.3. A Design Example................................ 158 4.2.4. Greenberger Approximation to the HR Algorithm....... 161

Cootents xi 4.3. The Weldon Algorithm.................................. 162 4.4. The Chase Algorithm.................................... 167 4.4.1. Standard Chase Algorithms......................... 167 4.4.2. Variations on the Chase Algorithms................... 169 4.5. Information-Set-Decoding Algorithms...................... 172 4.5.1. Information-Set-Only Algorithms.................... 172 4.5.2. Omura-Type Algorithms........................... 175 4.5.3. Partial Syndrome Decoding......................... 177 4.5.4. Some Performance Predictions....................... 178 Problems.......................................... I 79 5. Algebraic Techniques for Multiple Error Correction............. 181 5.1. Finite-Field Transforms.................................. 183 5.2. BCH Codes........................................... 185 5.3. BCH Decoding Techniques............................... 188 5.3.1. Frequency Domain Decoding... 188 5.3.2. Time Domain Decoding............................ 191 5.4. Solution of the Key Equation............................. 195 5.4.1. Euclid's Algorithm................................ 195 5.4.2. Berlekamp's Algorithm............................ 201 5.5. Implementation Considerations........................... 209 5.6. Correction of Errors and Erasures......................... 214 5.7. Performance Results.................................... 219 5.8. Remarks............................................. 222 Problems................................................ 224 6. Convolutional Code Structure and Viterbi Decoding............. 227 6.1. Binary Rate- I /2 Convolutional Codes...................... 228 6.2. Viterbi Decoding Algorithm.............................. 231 6.3. Rate-m/n Convolutional Codes............................ 235 6.4. Finite-State Machine Description and Distance Properties of Convolutional Codes.................................... 238 6.5. Performance of Convolutional Codes with Viterbi Decoding..... 242 6.5.1. Union Bounds................................... 243 6.5.2. Optimum Codes.................................. 246 6.5.3. Performance with Coherent PSK..................... 247 6.5.4. Performance with Orthogonal Signaling and Noncoherent Detection.... 6.6. Implementation Considerations.... 6.6.1. Synchronizer.... 6.6.2. Branch Metric Computer.... 6.6.3. Path Metric Storage and Updating.... 6.6.4. Storage and Updating of Hypothesized Information Sequences.............................. 6.6.5. Output Device.... 6.6.6. Demodulator Quantization and AGC.... 6.7. Remarks.................................. Problems.... 250 253 255 256 258 261 263 264 265 266

xii Contents 7. Other Convolutional Decoding Techniques..................... 267 7.1. Syndrome Decoding Techniques........................... 268 7.1.1. Basic Concepts................................... 268 7.1.2. Table Look-up Decoding........................... 278 7.1.3. Threshold Decoding............................... 283 7.2. Sequential Decoding Techniques........................... 297 7.2.1. Sequential Decoding via the Fano Algorithm............ 299 7.2.2. Selection of Sequential Decoding Metric............... 302 7.2.3. Code Selection................................... 304 7.2.4. Sequential Decoder Computational Problem............ 304 7.2.5. Performance of Sequential Decoders.................. 313 7.2.6. Implementation Considerations...................... 318 7.2.7. Stack Sequential Decoding.. ~~....................... 326 Problems 328 8. System Applications...................................... 331 8.1. Concatenated Codes.................................... 331 8.1.1. Concatenated Coding Concepts...................... 332 8.1.2. Reed-Solomon/Orthogonal Code Systems............. 333 8.1.3. Reed-Solomon/Short-Block Code Systems............. 337 8.1.4. Reed-Solomon/Convolutional Code Systems........... 339 8.2. Coding for the White Gaussian Noise Channel................ 341 8.3. Interleaver Structures for Coded Systems.................... 345 8.3.1. Periodic Interleavers.............................. 346 8.3.2. Pseudorandom Interleavers......................... 349 8.4. Coding for Burst Noise Channels.......................... 352 8.4.1. Burst Noise Processes............................. 353 8.4.2. Convolutional Code Performance with Random Erasures.. 354 8.4.3. Convolutional Code Performance with Periodic Burst Erasures........................................ 358 8.4.4. Performance Degradation Due to Random Erasure Burst.. 363 8.4.5. Interleaving Implications........................... 364 8.5. Coding for Spread Spectrum Systems....................... 366 8.5.1. Direct Sequence PN Spread Systems.................. 367 8.5.2. Frequency Hop Systems............................ 375 8.6. Coding for Bandwidth-Constrained Channels................. 381 Problems................................................. 390 Appendix A. Code Generators for BCH Codes.................... 393 Appendix B. Code Generators for Convolutional Codes............. 399 B.l. Viterbi Decoding....................................... 399 B.2. Table Look-up Decoding................................ 400 8.3. Threshold Decoding.................................... 400 B.4. Sequential Decoding.................................... 401 References.................................................. 409 Index... 415

Error-Correction Coding for Digital Communications