Digital Communications: An Overview of Fundamentals

Size: px

Start display at page:

Download "Digital Communications: An Overview of Fundamentals"

Meagan Moore
5 years ago
Views:

1 IMPERIAL COLLEGE of SCIENCE, TECHNOLOGY and MEDICINE, DEPARTMENT of ELECTRICAL and ELECTRONIC ENGINEERING. COMPACT LECTURE NOTES on COMMUNICATION THEORY. Prof Athanassios Manikas, Spring 2001 Digital Communications: An Overview of Fundamentals Aims: To provide the foundations of a typical Digital Communication System (DCS), in a block-diagram structure, and overviewing its operation. To define the main performance criteria for DCS and the associated parameter planes with emphasis given to the energy utilisation efficiency and bandwidth utilisation efficiency. To identify the theoretical limits on the performance of DCS To highlight system trade-offs. Information Sources

2 1. Introduction Wireline and fiber communications as well as wireless communications are fully digital. The general block structure of a Digital Communication System is shown in the following page and it is common practice its quality to be expressed in terms of the accuracy with which the binary digits delivered at the output of the detector (point B2 ^ ) represent the binary digits that were fed into the digital modulator (point B2). It is generally taken that it is the fraction of the binary digits that are delivered back in error that is a measure of the quality of the communication system. This fraction, or rate, is referred to as the bit error probability : /, or, Bit-Error-Rate BER (point B2 ^ ). Digital Communications - An Overview of Fundamentals 2 GENERAL BLOCK STRUCTURE OF A DIGITAL COMMUNICATION SYSTEM Con t. Info Source g(t) Sampler ( ) Fs N.B.:Fs>2Fg uantizer (, ) Source Encoder Discrete Channel Encoder Interleaver Channel Encoder Digital Modulator (M, Tcs) Analogue CHANNEL rcs=1/tcs bauds B k + 1 B H( f) f n(t) i Con t. Info Sink ^ g(t) 0 n(t) 0 SNRout =f( ) pe LPF -F g 1 +F g f ^ Source Decoder pe=f( EUE) ^ ^ ^ Discrete Channel Decoder DeInterleaver Digital Demodulator (M, Tcs) ^ (B,C) EUE BUE SNIRin Channel Decoder Discrete channel Digital Communications - An Overview of Fundamentals 3

3 1. Information Sources ì Information sources (or communication sources, or simply sources Ñ can be classified as ˆ Discrete, or ˆ Continuous ì The 'inverse' of an information/communication source is an information/communication sink (discrete or continuous) N.B. - terminology: 'continuous' 'analogue' Digital Communications - An Overview of Fundamentals Continuous Sources/Signals (Point-A) ìa source whose output 1Ð>Ñ is continuous signal in both amplitude and time is called a Continuous Source. ìa continuous source Ðwhich relates directly to analogue signal waveforms Ñ is described by its ensemble 1Ð>Ñß pdf ÐgÑwhere pdf ÐgÑ denotes the amplitude probability density function 1 1 of 1Ð>ÑÞ However, in order to describe fully a continuous source 1Ð>Ñ, in addition to the source ensemble, the power spectral density PSD 1 Ð0Ñ of the signal at the output has to be specified while its bandwidth F should also be identified. Digital Communications - An Overview of Fundamentals 5

4 1.2. Discrete Source (Point A2): ìa source which produces a sequence Ö\Ð8Ñ of symbols one after another, with each symbol being drawn from a finite alphabet \ œ ÖB" ß B# ß ÞÞÞß B with a rate < symbols/sec, and in which each symbol B \ is produced at the output of the \ 7 source with some associated probability PrÐB7Ñ, abbreviated : 7, i.e. PrÐB7Ñ œ : 7, is called a Discrete Source. ìif successive outputs from a discrete source are statistically independent, or in other words, if at each instant of time the source chooses for transmission one symbol from the set \ œ ÖB" ß B# ß ÞÞÞß B and its choices are independent from one time instant to the next, then the source is called a Discrete Memoryless Source ÐDMS Ñ. ìif : represents the vector with elements the probabilities associated with the symbols B7 \ of a source, then the set Ð\ß:Ñ ÐB ß : Ñ " " ÐB ß : Ñ # # Ð\ ß:Ñ œ with : 7 œ " ÞÞÞÞ 7œ" ÐBß : Ñ is defined as the Source Ensemble Digital Communications - An Overview of Fundamentals 6 ì A DMS source can be fully described by its ensemble Ð\ß :Ñ and its symbol rate (symbols per second) < \ The point A# may be considered as the input of a Digital Communication System where messages consist of sequences of "symbols" selected from an alphabet e.g. levels of a quantizer or telegraph letters, numbers and punctuations. Digital Communications - An Overview of Fundamentals 7

5 1.3. Measure of Information Generated by a Source Source Entropy ì Consider a discrete memoryless information source ÐB ß: Ñ " " ÐB# ß: # Ñ Ð\ ß: Ñ œ with : 7 œ " ÞÞÞÞ 7œ" ÐB ß: Ñ Then, the average information per symbol generated by the source is given by the so-called entropy of the source i.e. H\ œ H \ Ð: Ñ : 7.I ÐB 7 Ñ œ : 7. log # : 7 7œ" 7œ" Digital Communications - An Overview of Fundamentals 8 bits symbol ì H \ is a measure of the a priori uncertainty associated with the source output or, equivalently, a measure of the information obtained when the source output is observed. ì The notion of entropy is not restricted to the case where the ensemble is finite i.e. may be. It can also be shown that H \ is the minimum number of binary digits Ðbits Ñ needed to encode the source output. ì Based on entropy, the average information bit-rate output of the source is defined as follows: at the < < inf.h \ \ bits sec Digital Communications - An Overview of Fundamentals 9

6 ì Thus, we have two types of 'bits' and, therefore, two types of 'rates'. That is, data,3>=,3>= =/- =/- ˆ data rate œ <, in or simply info,3>=,3>= ˆ info rate œ < inf in =/- or simply =/- NB: ˆ In general: < inf Ÿ <, ˆ In ideal systems: < inf œ <, Digital Communications - An Overview of Fundamentals 10 ì Example if : data rate œ<, œ< B œ"! PrÐ!Ñ!Þ&,3> info,3> = = thenh =1 =1 PrÐ"Ñ!Þ& i.e. < <, œ inf X = ymbol data,3> info rate œ< œ< H œ"! inf B X,3>= =/-,3>= =/- Digital Communications - An Overview of Fundamentals 11

7 if : PrÐ"Ñ!Þ3 i.e. < <, inf X info rate,3>,3> = ymbol data,3> œ< inf œ< BHX œ 8.813,3>=,3>= data rate œ<, œ< B œ"! =/- PrÐ!Ñ!Þ7 s info = = thenh = = =/- Digital Communications - An Overview of Fundamentals A 'Joint' Source ì Consider two information sources with the ensembles Ð\ß :Ñ and Ð] ß ;Ñ, respectively, defined as follows: Ð\ß :Ñ œ ÐB ß : Ñ " " ÐB# ß :# Ñ ÞÞÞÞ ÐB ß : Ñ œ B7,Pr ÐB 7Ñ a7à"ÿ7ÿ œ: 7 Ð] ß ;Ñ œ ÐC ß ; Ñ " " ÐC# ß ;# Ñ ÞÞÞÞ ÐC ß ; Ñ O O œ C5,Pr ÐCÑ 5 a5à"ÿ5ÿo œ; 5 Digital Communications - An Overview of Fundamentals 13

8 ìlet us form a joined source where its symbols are taken to be pairs of symbols drawn from the two original sources \ and ]. The new source with alphabet Ö B ß C Ñß ÐB ß C Ñß ÞÞÞß ÐB ß C Ñ ( " " " # O is known as joint source defined as follows and its ensemble as joint ensemble Ð\ ], Ñ= ÐB 7ßC5Ñ,Pr ÐB7ßC5Ñ a75à"ÿ7ÿß"ÿ5ÿo œn 57 Digital Communications - An Overview of Fundamentals 14 Note that the joint probabilistic relationship between the symbols of \ and ], described by the so-called jointprobability matrix, œ PrÐB", C" Ñß PrÐB", C# Ñß ÞÞÞß PrÐB", COÑ PrÐB#, C" Ñ, PrÐB#, C# Ñ, ÞÞÞß PrÐB#, COÑ ÞÞÞ ÞÞÞ ÞÞÞ ÞÞÞ PrÐB, C Ñ, PrÐB, C Ñ, ÞÞÞ PrÐB, C Ñ " # O X Digital Communications - An Overview of Fundamentals 15

9 ì Digital Communications - An Overview of Fundamentals 16 ì The entropies associated with the above three sources \ß ] and \ ] are defined as follows: X H :. log : œ: log : \ 7 # 7 # 7œ" O X ] 5 log# 5 log# 5œ" O \ ] N57 N57 7œ" 5œ" H :. : œ; ; bits symbol bits symbol X H. log # œ 1 log # 1 O bits symbol where œ N ß N ß ÞÞÞß N N, N, ÞÞÞß N ÞÞÞ ÞÞÞ ÞÞÞ ÞÞÞ N ß N ß ÞÞÞ N "" "# " #" ## # O" O# O and 1 = " " ÞÞÞ " œ vectorof M ones Digital Communications - An Overview of Fundamentals 17

10 2. A Note on Information Sinks ì A communication sink is the destination of the symbols produced by a source and transmitted from the input to the output of a channel. It has one input and no output and it can be seen as the inverse of a source while it is described by 1. a performance (fidelity) criterion 3Ð\ß ] Ñ (e.g. criterion:fivñ 2. a maximum allowed distortion H 7+B $ $ (e.g. D œ"! i.e. FIV<10 Ñ 7+B Digital Communications - An Overview of Fundamentals Digital Communications - An Overview of Fundamentals 19

11 5) Markov Information Sources: Markov sources are modelled by Markov state transition diagrams and their entropy is given as follows: H œ N i=1 1 i.h bits/symbol i Rœnumber of states where 1 i œ probability to be on the i-th state H œ entropy of the i-th state Ðconsidered as a DMSÑ i Digital Communications - An Overview of Fundamentals 20 Example: Consider a two-state Markov source which gives the symbols A,B,C according to the following model: (A, ¾) (C, ¼) 1 2 (C, ¼) (B, ¾) Then H =? (for you) 1 H =? 2 # i= " and H= 1.H bits/symbol œ? i i Digital Communications - An Overview of Fundamentals 21

12 RÞFÞ À Digital Communications - An Overview of Fundamentals 22

Information Sources. Professor A. Manikas. Imperial College London. EE303 - Communication Systems An Overview of Fundamentals

Information Sources. Professor A. Manikas. Imperial College London. EE303 - Communication Systems An Overview of Fundamentals Information Sources Professor A. Manikas Imperial College London EE303 - Communication Systems An Overview of Fundamentals Prof. A. Manikas (Imperial College) EE303: Information Sources 24 Oct. 2011 1