The information loss in quantization
|
|
- Melinda Chambers
- 5 years ago
- Views:
Transcription
1 The information loss in quantization The rough meaning of quantization in the frame of coding is representing numerical quantities with a finite set of symbols. The mapping between numbers, which are normally real values, and the corresponding symbols from a finite set is an irreversable operation because it is a many-to-one mapping. A symbol at the quantizer output can be generated with many inputs which have similar values. In this scheme, the exact information of x(t) is lost, and only an approximation of x(t) can be recovered from y(t) using an inverse operation of the quantizer. The amount of loss due to the quantizer is represented by where d(.) is a distance measure. Usually or. We will call this difference quantization error. A very common class of quantizers are known as "scalar quantizers". Scalar quantizers have a very easy interpretation in the form of algebraic analysis. First, let's start with the well known rounding operator as a quantization operation:. The operator yields integer numbers which are nearest to the number, x. Obviously, we cannot totally recover x from q(x). More generally, we describe a scalar quantizer by A list of output levels (will be called codebook or reproduction values ) A set of disjoint intervals Whenever a value is numerically in an interval the quantizer output becomes. Quantizer block, therefore, can be analyzed in two components: Encoder: and or an equivalent set of of binary vectors. Decoder: Overall Quantizer: and the reproduction codebook is
2 . The coding efficiency of the quantizer comes from the fact that, if you have a total of M quantization levels, then each symbol can be represented by at most bits (usually less bits per symbol can be obtained using other compression techniques). Normally, a high precision real valued number should be represented with infinite (or very high number of) bits, therefore there is a compression. Attention: After the binary representation of the symbol the decompressor, which tries to reconstruct the data from the compressed bit-stream, should also know all of the quantization values (or the codebook ). This is true for most of the cases since the encoders and decoders both behave on a pre-determined compression algorithm. The number of these levels or intervals determine the precision of the quantizer. Let us assume that i=1..m, so there will be M output levels. Formally, the overall relation can be written as for "Obviously, the biggest issue is to select the intervals and reproduction levels the best way to minimize the quantization error at a given number of quantization levels, M." The simplest quantizer is the uniform scalar quantizer. The rounding operator that we have mentioned above is a special uniform scalar quantizer with the intervals and the output levels. Consider the figure below. The illustrated quantizer quantizes the input signal x(n) using uniform quantization steps to produce y(n). As you see, y(n) is only an approximate versron of the original x(n), and some of the precision is lost. The input-output caracteristics of such a scalar quantizer can be illustrated as:
3 The above figure shows a uniform scalar quantizer with M=8 levels and interval width of. If a large interval around zero is quantized to value zero, the quantizer is called a deadzone quantizer. An example is: The design issue for a quantizer is to minimize the reproduction error. We will come back to the design concepts later, however you should start thinking about ways to make the information loss as small as possible. Using the above error notation, we can model the quantization error as an additive noise on top of the signal x, therefore the quantizer output is. The graphical chart is:
4 The optimization of the quantizer by minimizing the error generally produces a nonuniform quantizer. In the non-uniform case, the intervals are not the same for all i. Let us denote the general i-th interval as. In this notation are the thresholds which form an increasing sequence. We can say that the width of the i-th interval is. Let us also denote the output of the quantizer to an input in the i- th interval as. A general non-uniform quantizer may look like: Notice that the first interval starts from -infinity and the last interval ends at +infinity. The quality of the quantizer is measured by the amount of error where d(.) is a distance measure as described above. The average error (which will be called distortion) can be written as (Eq. 1) using the fact that the input signal is treated as a random signal. The E{.} operator means the expected value of the argument. Attention: If you are uncomfortable with expected values and random variables, please refer to any book about probability and random processes. You can find a very brief review with basic properties about random processes and random signals here. From the definition of expected value, we can write: (Eq. 2) where is the Probability Distribution Function (PDF) of the random variable x. Minimizing the distortion D will maximize the reproduction quality.
5 In practice, you may not know the PDF of the input, and you may only observe L samples from the data. In that case, the distortion can be approximated by. The distortion expression in Eq. 1 and Eq. 2 can be formed into an addition of individual distortions in separate intervals: (Eq. 3) where, distortion measure is (MSE). is the probability of being in the i-th interval. If the then the distortion is called the Mean Squared Error Exercise 1: Find the MSE of the level uniform quantizer formed by selecting the interval widths as and the quantization values equal to the mid-points of the intervals. Assume that a random signal with uniform PDF in the range is fed to the quantizer. (This is also known as R-bit uniform quantizer). Answer: The MSE is Eq. 4 This is an important result, and the MSE found for the above example approximately holds for uniform quantizers applied to signals with different PDF's if the quantization intervals are taken small. When the intervals are small enough, one can assume that the PDF is almost uniform within that small range, and the above distortion expression holds. Homework 1: In the above example, the uniform quantizer was acting on a random
6 signal whose PDF is uniform between -X max and X max. The quantization interval was and the quantiaztion output was the midpoint of each interval : If input was between (i-1) and i (corresponding to the ith interval) then the quantizer output was. We saw that the quantization error (, when ) has a power (called MSE) of. The uniform PDF is illustrated at the left part of the figure below. For the homework, consider the same quantizer, but this time the input signal has a PDF as shown at the right part of the figure below (the triangle shaped). The quantizer again uses interval width of and output levels as the mid point of the interval (just like before). What is the MSE of the quantizer to this input? Another important consequence of the above result is about the relation between the amount of bits (R) and distortion D. Clearly, D is proportional to and is inversely proportional to R. Precisely,. This relation implies that if you increase R, the distortion is decreased. This is a famous result, know as the rate-distortion function in the information theory. More precisely, we define which is the minimum attainable distortion at a given rate R. The plot of versus R is known as the rate-distortion curve. Practically, a quantizer distortion at a given R is above this curve. A good quantizer produces distortions at given rates R as near to the rate-distortion curve as possible. An alternative definition of is and the minimum is over all codebooks (or reproduction levels and all interval partitions. The typical characteristics of a rate-distortion curve and a practical quantizer performance is illustrated as follows:
7 Exercise 2: A sinusoidal signal with peak-to-peak amplitude 5V is quantized by a R- bit Analog-to-Digital Convertor (ADC). Assuming the quantization intervals are uniform, determine: 1. The quantization step size. 2. The root-mean-squared (RMS) signal to noise (due to quantization) ratio SQNR. 3. The SQNR for a 16 bit ADC. Answer: The quantiaztion step size is: We know that the signal power fo a sinusoid with amplitude A is given as the above step size, the SQNR (in decibels) is given as:.using which indicates that each aditional bit adds to the SQNR performance by approximately 6dB - known as the 6dB per bit rule. Using this result, the 16 bit ADC gives SQNR of 6.02x =98dB (CD quality). In the following sections we will elaborate more on quantization, its techniques and variations. However, I encourage you to keep in mind where in the coding framawork we are this week. The basic steps of a typical coder was given in the last week's introduction. This week, we are covering the typical middle block of the coding scheme, called quantization, where the loss and bit reductions occur. Before the quantization, there is usually a predictive coder or a transform coder block and after the quantization, there is usually an entropy coder to compress the bits produced by the quantizer.
8 Now that we have an understanding about how much information loss we have for a quantizer at a given rate, we can proceed to the quantizer design techniques.
at Some sort of quantization is necessary to represent continuous signals in digital form
Quantization at Some sort of quantization is necessary to represent continuous signals in digital form x(n 1,n ) x(t 1,tt ) D Sampler Quantizer x q (n 1,nn ) Digitizer (A/D) Quantization is also used for
More informationMultimedia Systems Giorgio Leonardi A.A Lecture 4 -> 6 : Quantization
Multimedia Systems Giorgio Leonardi A.A.2014-2015 Lecture 4 -> 6 : Quantization Overview Course page (D.I.R.): https://disit.dir.unipmn.it/course/view.php?id=639 Consulting: Office hours by appointment:
More informationMultimedia Communications. Scalar Quantization
Multimedia Communications Scalar Quantization Scalar Quantization In many lossy compression applications we want to represent source outputs using a small number of code words. Process of representing
More informationPrinciples of Communications
Principles of Communications Weiyao Lin, PhD Shanghai Jiao Tong University Chapter 4: Analog-to-Digital Conversion Textbook: 7.1 7.4 2010/2011 Meixia Tao @ SJTU 1 Outline Analog signal Sampling Quantization
More informationScalar and Vector Quantization. National Chiao Tung University Chun-Jen Tsai 11/06/2014
Scalar and Vector Quantization National Chiao Tung University Chun-Jen Tsai 11/06/014 Basic Concept of Quantization Quantization is the process of representing a large, possibly infinite, set of values
More information7.1 Sampling and Reconstruction
Haberlesme Sistemlerine Giris (ELE 361) 6 Agustos 2017 TOBB Ekonomi ve Teknoloji Universitesi, Guz 2017-18 Dr. A. Melda Yuksel Turgut & Tolga Girici Lecture Notes Chapter 7 Analog to Digital Conversion
More informationCMPT 889: Lecture 3 Fundamentals of Digital Audio, Discrete-Time Signals
CMPT 889: Lecture 3 Fundamentals of Digital Audio, Discrete-Time Signals Tamara Smyth, tamaras@cs.sfu.ca School of Computing Science, Simon Fraser University October 6, 2005 1 Sound Sound waves are longitudinal
More informationGaussian source Assumptions d = (x-y) 2, given D, find lower bound of I(X;Y)
Gaussian source Assumptions d = (x-y) 2, given D, find lower bound of I(X;Y) E{(X-Y) 2 } D
More informationFinite Word Length Effects and Quantisation Noise. Professors A G Constantinides & L R Arnaut
Finite Word Length Effects and Quantisation Noise 1 Finite Word Length Effects Finite register lengths and A/D converters cause errors at different levels: (i) input: Input quantisation (ii) system: Coefficient
More informationEE368B Image and Video Compression
EE368B Image and Video Compression Homework Set #2 due Friday, October 20, 2000, 9 a.m. Introduction The Lloyd-Max quantizer is a scalar quantizer which can be seen as a special case of a vector quantizer
More informationCoding for Discrete Source
EGR 544 Communication Theory 3. Coding for Discrete Sources Z. Aliyazicioglu Electrical and Computer Engineering Department Cal Poly Pomona Coding for Discrete Source Coding Represent source data effectively
More informationC.M. Liu Perceptual Signal Processing Lab College of Computer Science National Chiao-Tung University
Quantization C.M. Liu Perceptual Signal Processing Lab College of Computer Science National Chiao-Tung University http://www.csie.nctu.edu.tw/~cmliu/courses/compression/ Office: EC538 (03)5731877 cmliu@cs.nctu.edu.tw
More informationBasic Principles of Video Coding
Basic Principles of Video Coding Introduction Categories of Video Coding Schemes Information Theory Overview of Video Coding Techniques Predictive coding Transform coding Quantization Entropy coding Motion
More informationImage Compression. Fundamentals: Coding redundancy. The gray level histogram of an image can reveal a great deal of information about the image
Fundamentals: Coding redundancy The gray level histogram of an image can reveal a great deal of information about the image That probability (frequency) of occurrence of gray level r k is p(r k ), p n
More informationQuantization. Introduction. Roadmap. Optimal Quantizer Uniform Quantizer Non Uniform Quantizer Rate Distorsion Theory. Source coding.
Roadmap Quantization Optimal Quantizer Uniform Quantizer Non Uniform Quantizer Rate Distorsion Theory Source coding 2 Introduction 4 1 Lossy coding Original source is discrete Lossless coding: bit rate
More informationMultimedia & Computer Visualization. Exercise #5. JPEG compression
dr inż. Jacek Jarnicki, dr inż. Marek Woda Institute of Computer Engineering, Control and Robotics Wroclaw University of Technology {jacek.jarnicki, marek.woda}@pwr.wroc.pl Exercise #5 JPEG compression
More informationETSF15 Analog/Digital. Stefan Höst
ETSF15 Analog/Digital Stefan Höst Physical layer Analog vs digital Sampling, quantisation, reconstruction Modulation Represent digital data in a continuous world Disturbances Noise and distortion Synchronization
More informationEE-597 Notes Quantization
EE-597 Notes Quantization Phil Schniter June, 4 Quantization Given a continuous-time and continuous-amplitude signal (t, processing and storage by modern digital hardware requires discretization in both
More information4. Quantization and Data Compression. ECE 302 Spring 2012 Purdue University, School of ECE Prof. Ilya Pollak
4. Quantization and Data Compression ECE 32 Spring 22 Purdue University, School of ECE Prof. What is data compression? Reducing the file size without compromising the quality of the data stored in the
More informationThe Secrets of Quantization. Nimrod Peleg Update: Sept. 2009
The Secrets of Quantization Nimrod Peleg Update: Sept. 2009 What is Quantization Representation of a large set of elements with a much smaller set is called quantization. The number of elements in the
More informationWavelet Scalable Video Codec Part 1: image compression by JPEG2000
1 Wavelet Scalable Video Codec Part 1: image compression by JPEG2000 Aline Roumy aline.roumy@inria.fr May 2011 2 Motivation for Video Compression Digital video studio standard ITU-R Rec. 601 Y luminance
More informationSCALABLE AUDIO CODING USING WATERMARKING
SCALABLE AUDIO CODING USING WATERMARKING Mahmood Movassagh Peter Kabal Department of Electrical and Computer Engineering McGill University, Montreal, Canada Email: {mahmood.movassagh@mail.mcgill.ca, peter.kabal@mcgill.ca}
More informationEE67I Multimedia Communication Systems
EE67I Multimedia Communication Systems Lecture 5: LOSSY COMPRESSION In these schemes, we tradeoff error for bitrate leading to distortion. Lossy compression represents a close approximation of an original
More informationEEO 401 Digital Signal Processing Prof. Mark Fowler
EEO 401 Digital Signal Processing Pro. Mark Fowler Note Set #14 Practical A-to-D Converters and D-to-A Converters Reading Assignment: Sect. 6.3 o Proakis & Manolakis 1/19 The irst step was to see that
More informationOn Compression Encrypted Data part 2. Prof. Ja-Ling Wu The Graduate Institute of Networking and Multimedia National Taiwan University
On Compression Encrypted Data part 2 Prof. Ja-Ling Wu The Graduate Institute of Networking and Multimedia National Taiwan University 1 Brief Summary of Information-theoretic Prescription At a functional
More informationModern Digital Communication Techniques Prof. Suvra Sekhar Das G. S. Sanyal School of Telecommunication Indian Institute of Technology, Kharagpur
Modern Digital Communication Techniques Prof. Suvra Sekhar Das G. S. Sanyal School of Telecommunication Indian Institute of Technology, Kharagpur Lecture - 15 Analog to Digital Conversion Welcome to the
More informationNumber Representation and Waveform Quantization
1 Number Representation and Waveform Quantization 1 Introduction This lab presents two important concepts for working with digital signals. The first section discusses how numbers are stored in memory.
More informationExample: for source
Nonuniform scalar quantizer References: Sayood Chap. 9, Gersho and Gray, Chap.'s 5 and 6. The basic idea: For a nonuniform source density, put smaller cells and levels where the density is larger, thereby
More informationCommunication Engineering Prof. Surendra Prasad Department of Electrical Engineering Indian Institute of Technology, Delhi
Communication Engineering Prof. Surendra Prasad Department of Electrical Engineering Indian Institute of Technology, Delhi Lecture - 41 Pulse Code Modulation (PCM) So, if you remember we have been talking
More informationELEN E4810: Digital Signal Processing Topic 11: Continuous Signals. 1. Sampling and Reconstruction 2. Quantization
ELEN E4810: Digital Signal Processing Topic 11: Continuous Signals 1. Sampling and Reconstruction 2. Quantization 1 1. Sampling & Reconstruction DSP must interact with an analog world: A to D D to A x(t)
More informationPulse-Code Modulation (PCM) :
PCM & DPCM & DM 1 Pulse-Code Modulation (PCM) : In PCM each sample of the signal is quantized to one of the amplitude levels, where B is the number of bits used to represent each sample. The rate from
More informationCS578- Speech Signal Processing
CS578- Speech Signal Processing Lecture 7: Speech Coding Yannis Stylianou University of Crete, Computer Science Dept., Multimedia Informatics Lab yannis@csd.uoc.gr Univ. of Crete Outline 1 Introduction
More informationLecture 2: Introduction to Audio, Video & Image Coding Techniques (I) -- Fundaments
Lecture 2: Introduction to Audio, Video & Image Coding Techniques (I) -- Fundaments Dr. Jian Zhang Conjoint Associate Professor NICTA & CSE UNSW COMP9519 Multimedia Systems S2 2006 jzhang@cse.unsw.edu.au
More informationQuantisation. Uniform Quantisation. Tcom 370: Principles of Data Communications University of Pennsylvania. Handout 5 c Santosh S.
Tcom 370: Principles of Data Communications Quantisation Handout 5 Quantisation involves a map of the real line into a discrete set of quantisation levels. Given a set of M quantisation levels {S 0, S
More informationLecture 2: Introduction to Audio, Video & Image Coding Techniques (I) -- Fundaments. Tutorial 1. Acknowledgement and References for lectures 1 to 5
Lecture : Introduction to Audio, Video & Image Coding Techniques (I) -- Fundaments Dr. Jian Zhang Conjoint Associate Professor NICTA & CSE UNSW COMP959 Multimedia Systems S 006 jzhang@cse.unsw.edu.au Acknowledgement
More informationMidterm Exam Information Theory Fall Midterm Exam. Time: 09:10 12:10 11/23, 2016
Midterm Exam Time: 09:10 12:10 11/23, 2016 Name: Student ID: Policy: (Read before You Start to Work) The exam is closed book. However, you are allowed to bring TWO A4-size cheat sheet (single-sheet, two-sided).
More informationAnalog to Digital Converters (ADCs)
Analog to Digital Converters (ADCs) Note: Figures are copyrighted Proakis & Manolakis, Digital Signal Processing, 4 th Edition, Pearson Publishers. Embedded System Design A Unified HW Approach, Vahid/Givargis,
More informationFACULTY OF ENGINEERING MULTIMEDIA UNIVERSITY LAB SHEET
FACULTY OF ENGINEERING MULTIMEDIA UNIVERSITY LAB SHEET ETM 3136 Digital Communications Trimester 1 (2010/2011) DTL1: Pulse Code Modulation (PCM) Important Notes: Students MUST read this lab sheet before
More informationDigital communication system. Shannon s separation principle
Digital communication system Representation of the source signal by a stream of (binary) symbols Adaptation to the properties of the transmission channel information source source coder channel coder modulation
More informationOn Optimal Coding of Hidden Markov Sources
2014 Data Compression Conference On Optimal Coding of Hidden Markov Sources Mehdi Salehifar, Emrah Akyol, Kumar Viswanatha, and Kenneth Rose Department of Electrical and Computer Engineering University
More informationSeminar: D. Jeon, Energy-efficient Digital Signal Processing Hardware Design Mon Sept 22, 9:30-11:30am in 3316 EECS
EECS 452 Lecture 6 Today: Announcements: Rounding and quantization Analog to digital conversion Lab 3 starts next week Hw3 due on tuesday Project teaming meeting: today 7-9PM, Dow 3150 My new office hours:
More informationReview of Quantization. Quantization. Bring in Probability Distribution. L-level Quantization. Uniform partition
Review of Quantization UMCP ENEE631 Slides (created by M.Wu 004) Quantization UMCP ENEE631 Slides (created by M.Wu 001/004) L-level Quantization Minimize errors for this lossy process What L values to
More informationFuzzy quantization of Bandlet coefficients for image compression
Available online at www.pelagiaresearchlibrary.com Advances in Applied Science Research, 2013, 4(2):140-146 Fuzzy quantization of Bandlet coefficients for image compression R. Rajeswari and R. Rajesh ISSN:
More informationECE533 Digital Image Processing. Embedded Zerotree Wavelet Image Codec
University of Wisconsin Madison Electrical Computer Engineering ECE533 Digital Image Processing Embedded Zerotree Wavelet Image Codec Team members Hongyu Sun Yi Zhang December 12, 2003 Table of Contents
More informationDigital Signal Processing 2/ Advanced Digital Signal Processing Lecture 3, SNR, non-linear Quantisation Gerald Schuller, TU Ilmenau
Digital Signal Processing 2/ Advanced Digital Signal Processing Lecture 3, SNR, non-linear Quantisation Gerald Schuller, TU Ilmenau What is our SNR if we have a sinusoidal signal? What is its pdf? Basically
More informationCompression and Coding
Compression and Coding Theory and Applications Part 1: Fundamentals Gloria Menegaz 1 Transmitter (Encoder) What is the problem? Receiver (Decoder) Transformation information unit Channel Ordering (significance)
More informationEE 5345 Biomedical Instrumentation Lecture 12: slides
EE 5345 Biomedical Instrumentation Lecture 1: slides 4-6 Carlos E. Davila, Electrical Engineering Dept. Southern Methodist University slides can be viewed at: http:// www.seas.smu.edu/~cd/ee5345.html EE
More informationSource Coding. Scalar Quantization
Source Coding Source Coding The Communications Toolbox includes some basic functions for source coding. Source coding, also known as quantization or signal formatting, includes the concepts of analog-to-digital
More informationModule 5 EMBEDDED WAVELET CODING. Version 2 ECE IIT, Kharagpur
Module 5 EMBEDDED WAVELET CODING Lesson 13 Zerotree Approach. Instructional Objectives At the end of this lesson, the students should be able to: 1. Explain the principle of embedded coding. 2. Show the
More informationModule 3 LOSSY IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur
Module 3 LOSSY IMAGE COMPRESSION SYSTEMS Lesson 7 Delta Modulation and DPCM Instructional Objectives At the end of this lesson, the students should be able to: 1. Describe a lossy predictive coding scheme.
More informationLecture 5 Channel Coding over Continuous Channels
Lecture 5 Channel Coding over Continuous Channels I-Hsiang Wang Department of Electrical Engineering National Taiwan University ihwang@ntu.edu.tw November 14, 2014 1 / 34 I-Hsiang Wang NIT Lecture 5 From
More informationDigital Image Processing Lectures 25 & 26
Lectures 25 & 26, Professor Department of Electrical and Computer Engineering Colorado State University Spring 2015 Area 4: Image Encoding and Compression Goal: To exploit the redundancies in the image
More informationAudio /Video Signal Processing. Lecture 2, Quantization, SNR Gerald Schuller, TU Ilmenau
Audio /Video Signal Processing Lecture 2, Quantization, SNR Gerald Schuller, TU Ilmenau Quantization Signal to Noise Ratio (SNR). Assume we have a A/D converter with a quantizer with a certain number of
More informationEE 230 Lecture 40. Data Converters. Amplitude Quantization. Quantization Noise
EE 230 Lecture 40 Data Converters Amplitude Quantization Quantization Noise Review from Last Time: Time Quantization Typical ADC Environment Review from Last Time: Time Quantization Analog Signal Reconstruction
More informationModule 2 LOSSLESS IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur
Module 2 LOSSLESS IMAGE COMPRESSION SYSTEMS Lesson 5 Other Coding Techniques Instructional Objectives At the end of this lesson, the students should be able to:. Convert a gray-scale image into bit-plane
More informationDesign of Optimal Quantizers for Distributed Source Coding
Design of Optimal Quantizers for Distributed Source Coding David Rebollo-Monedero, Rui Zhang and Bernd Girod Information Systems Laboratory, Electrical Eng. Dept. Stanford University, Stanford, CA 94305
More informationEE5356 Digital Image Processing
EE5356 Digital Image Processing INSTRUCTOR: Dr KR Rao Spring 007, Final Thursday, 10 April 007 11:00 AM 1:00 PM ( hours) (Room 111 NH) INSTRUCTIONS: 1 Closed books and closed notes All problems carry weights
More information1 Ex. 1 Verify that the function H(p 1,..., p n ) = k p k log 2 p k satisfies all 8 axioms on H.
Problem sheet Ex. Verify that the function H(p,..., p n ) = k p k log p k satisfies all 8 axioms on H. Ex. (Not to be handed in). looking at the notes). List as many of the 8 axioms as you can, (without
More informationTutorial Sheet #2 discrete vs. continuous functions, periodicity, sampling
2.39 utorial Sheet #2 discrete vs. continuous functions, periodicity, sampling We will encounter two classes of signals in this class, continuous-signals and discrete-signals. he distinct mathematical
More informationVector Quantization. Institut Mines-Telecom. Marco Cagnazzo, MN910 Advanced Compression
Institut Mines-Telecom Vector Quantization Marco Cagnazzo, cagnazzo@telecom-paristech.fr MN910 Advanced Compression 2/66 19.01.18 Institut Mines-Telecom Vector Quantization Outline Gain-shape VQ 3/66 19.01.18
More informationAnalysis of Finite Wordlength Effects
Analysis of Finite Wordlength Effects Ideally, the system parameters along with the signal variables have infinite precision taing any value between and In practice, they can tae only discrete values within
More informationIntroduction to digital systems. Juan P Bello
Introduction to digital systems Juan P Bello Analogue vs Digital (1) Analog information is made up of a continuum of values within a given range At its most basic, digital information can assume only one
More informationAudio Coding. Fundamentals Quantization Waveform Coding Subband Coding P NCTU/CSIE DSPLAB C.M..LIU
Audio Coding P.1 Fundamentals Quantization Waveform Coding Subband Coding 1. Fundamentals P.2 Introduction Data Redundancy Coding Redundancy Spatial/Temporal Redundancy Perceptual Redundancy Compression
More informationNotes 3: Stochastic channels and noisy coding theorem bound. 1 Model of information communication and noisy channel
Introduction to Coding Theory CMU: Spring 2010 Notes 3: Stochastic channels and noisy coding theorem bound January 2010 Lecturer: Venkatesan Guruswami Scribe: Venkatesan Guruswami We now turn to the basic
More informationMotivation for Arithmetic Coding
Motivation for Arithmetic Coding Motivations for arithmetic coding: 1) Huffman coding algorithm can generate prefix codes with a minimum average codeword length. But this length is usually strictly greater
More informationCompression methods: the 1 st generation
Compression methods: the 1 st generation 1998-2017 Josef Pelikán CGG MFF UK Praha pepca@cgg.mff.cuni.cz http://cgg.mff.cuni.cz/~pepca/ Still1g 2017 Josef Pelikán, http://cgg.mff.cuni.cz/~pepca 1 / 32 Basic
More informationX 1 : X Table 1: Y = X X 2
ECE 534: Elements of Information Theory, Fall 200 Homework 3 Solutions (ALL DUE to Kenneth S. Palacio Baus) December, 200. Problem 5.20. Multiple access (a) Find the capacity region for the multiple-access
More information1. Quantization Signal to Noise Ratio (SNR).
Digital Signal Processing 2/ Advanced Digital Signal Processing Lecture 2, Quantization, SNR Gerald Schuller, TU Ilmenau 1. Quantization Signal to Noise Ratio (SNR). Assume we have a A/D converter with
More informationDigital Signal Processing
COMP ENG 4TL4: Digital Signal Processing Notes for Lecture #3 Wednesday, September 10, 2003 1.4 Quantization Digital systems can only represent sample amplitudes with a finite set of prescribed values,
More informationEECS 229A Spring 2007 * * (a) By stationarity and the chain rule for entropy, we have
EECS 229A Spring 2007 * * Solutions to Homework 3 1. Problem 4.11 on pg. 93 of the text. Stationary processes (a) By stationarity and the chain rule for entropy, we have H(X 0 ) + H(X n X 0 ) = H(X 0,
More information18.2 Continuous Alphabet (discrete-time, memoryless) Channel
0-704: Information Processing and Learning Spring 0 Lecture 8: Gaussian channel, Parallel channels and Rate-distortion theory Lecturer: Aarti Singh Scribe: Danai Koutra Disclaimer: These notes have not
More informationMARKOV CHAINS A finite state Markov chain is a sequence of discrete cv s from a finite alphabet where is a pmf on and for
MARKOV CHAINS A finite state Markov chain is a sequence S 0,S 1,... of discrete cv s from a finite alphabet S where q 0 (s) is a pmf on S 0 and for n 1, Q(s s ) = Pr(S n =s S n 1 =s ) = Pr(S n =s S n 1
More informationModule 3. Quantization and Coding. Version 2, ECE IIT, Kharagpur
Module Quantization and Coding ersion, ECE IIT, Kharagpur Lesson Logarithmic Pulse Code Modulation (Log PCM) and Companding ersion, ECE IIT, Kharagpur After reading this lesson, you will learn about: Reason
More information- An Image Coding Algorithm
- An Image Coding Algorithm Shufang Wu http://www.sfu.ca/~vswu vswu@cs.sfu.ca Friday, June 14, 2002 22-1 Agenda Overview Discrete Wavelet Transform Zerotree Coding of Wavelet Coefficients Successive-Approximation
More informationEE 230 Lecture 43. Data Converters
EE 230 Lecture 43 Data Converters Review from Last Time: Amplitude Quantization Unwanted signals in the output of a system are called noise. Distortion Smooth nonlinearities Frequency attenuation Large
More informationPerformance Bounds for Joint Source-Channel Coding of Uniform. Departements *Communications et **Signal
Performance Bounds for Joint Source-Channel Coding of Uniform Memoryless Sources Using a Binary ecomposition Seyed Bahram ZAHIR AZAMI*, Olivier RIOUL* and Pierre UHAMEL** epartements *Communications et
More informationAnalog Digital Sampling & Discrete Time Discrete Values & Noise Digital-to-Analog Conversion Analog-to-Digital Conversion
Analog Digital Sampling & Discrete Time Discrete Values & Noise Digital-to-Analog Conversion Analog-to-Digital Conversion 6.082 Fall 2006 Analog Digital, Slide Plan: Mixed Signal Architecture volts bits
More informationRevision of Lecture 5
Revision of Lecture 5 Information transferring across channels Channel characteristics and binary symmetric channel Average mutual information Average mutual information tells us what happens to information
More informationIntroduction p. 1 Compression Techniques p. 3 Lossless Compression p. 4 Lossy Compression p. 5 Measures of Performance p. 5 Modeling and Coding p.
Preface p. xvii Introduction p. 1 Compression Techniques p. 3 Lossless Compression p. 4 Lossy Compression p. 5 Measures of Performance p. 5 Modeling and Coding p. 6 Summary p. 10 Projects and Problems
More informationON SCALABLE CODING OF HIDDEN MARKOV SOURCES. Mehdi Salehifar, Tejaswi Nanjundaswamy, and Kenneth Rose
ON SCALABLE CODING OF HIDDEN MARKOV SOURCES Mehdi Salehifar, Tejaswi Nanjundaswamy, and Kenneth Rose Department of Electrical and Computer Engineering University of California, Santa Barbara, CA, 93106
More informationDepartment of Electrical Engineering, Polytechnic University, Brooklyn Fall 05 EL DIGITAL IMAGE PROCESSING (I) Final Exam 1/5/06, 1PM-4PM
Department of Electrical Engineering, Polytechnic University, Brooklyn Fall 05 EL512 --- DIGITAL IMAGE PROCESSING (I) Y. Wang Final Exam 1/5/06, 1PM-4PM Your Name: ID Number: Closed book. One sheet of
More informationError Spectrum Shaping and Vector Quantization. Jon Dattorro Christine Law
Error Spectrum Shaping and Vector Quantization Jon Dattorro Christine Law in partial fulfillment of the requirements for EE392c Stanford University Autumn 1997 0. Introduction We view truncation noise
More informationUNIT V FINITE WORD LENGTH EFFECTS IN DIGITAL FILTERS PART A 1. Define 1 s complement form? In 1,s complement form the positive number is represented as in the sign magnitude form. To obtain the negative
More informationImage Compression using DPCM with LMS Algorithm
Image Compression using DPCM with LMS Algorithm Reenu Sharma, Abhay Khedkar SRCEM, Banmore -----------------------------------------------------------------****---------------------------------------------------------------
More informationPhysical Layer and Coding
Physical Layer and Coding Muriel Médard Professor EECS Overview A variety of physical media: copper, free space, optical fiber Unified way of addressing signals at the input and the output of these media:
More informationFault Tolerance Technique in Huffman Coding applies to Baseline JPEG
Fault Tolerance Technique in Huffman Coding applies to Baseline JPEG Cung Nguyen and Robert G. Redinbo Department of Electrical and Computer Engineering University of California, Davis, CA email: cunguyen,
More informationRelations and Equivalence Relations
Relations and Equivalence Relations In this section, we shall introduce a formal definition for the notion of a relation on a set. This is something we often take for granted in elementary algebra courses,
More informationPARAMETRIC coding has proven to be very effective
966 IEEE TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING, VOL. 15, NO. 3, MARCH 2007 High-Resolution Spherical Quantization of Sinusoidal Parameters Pim Korten, Jesper Jensen, and Richard Heusdens
More informationRoundoff Noise in Digital Feedback Control Systems
Chapter 7 Roundoff Noise in Digital Feedback Control Systems Digital control systems are generally feedback systems. Within their feedback loops are parts that are analog and parts that are digital. At
More informationAnalysis of methods for speech signals quantization
INFOTEH-JAHORINA Vol. 14, March 2015. Analysis of methods for speech signals quantization Stefan Stojkov Mihajlo Pupin Institute, University of Belgrade Belgrade, Serbia e-mail: stefan.stojkov@pupin.rs
More informationSIGNAL COMPRESSION Lecture Shannon-Fano-Elias Codes and Arithmetic Coding
SIGNAL COMPRESSION Lecture 3 4.9.2007 Shannon-Fano-Elias Codes and Arithmetic Coding 1 Shannon-Fano-Elias Coding We discuss how to encode the symbols {a 1, a 2,..., a m }, knowing their probabilities,
More informationProyecto final de carrera
UPC-ETSETB Proyecto final de carrera A comparison of scalar and vector quantization of wavelet decomposed images Author : Albane Delos Adviser: Luis Torres 2 P a g e Table of contents Table of figures...
More informationMultimedia Communications. Mathematical Preliminaries for Lossless Compression
Multimedia Communications Mathematical Preliminaries for Lossless Compression What we will see in this chapter Definition of information and entropy Modeling a data source Definition of coding and when
More informationLecture 20: Quantization and Rate-Distortion
Lecture 20: Quantization and Rate-Distortion Quantization Introduction to rate-distortion theorem Dr. Yao Xie, ECE587, Information Theory, Duke University Approimating continuous signals... Dr. Yao Xie,
More informationRun-length & Entropy Coding. Redundancy Removal. Sampling. Quantization. Perform inverse operations at the receiver EEE
General e Image Coder Structure Motion Video x(s 1,s 2,t) or x(s 1,s 2 ) Natural Image Sampling A form of data compression; usually lossless, but can be lossy Redundancy Removal Lossless compression: predictive
More informationWhat does such a voltage signal look like? Signals are commonly observed and graphed as functions of time:
Objectives Upon completion of this module, you should be able to: understand uniform quantizers, including dynamic range and sources of error, represent numbers in two s complement binary form, assign
More informationDiscrete Mathematics and Probability Theory Fall 2013 Vazirani Note 12. Random Variables: Distribution and Expectation
CS 70 Discrete Mathematics and Probability Theory Fall 203 Vazirani Note 2 Random Variables: Distribution and Expectation We will now return once again to the question of how many heads in a typical sequence
More information2. the basis functions have different symmetries. 1 k = 0. x( t) 1 t 0 x(t) 0 t 1
In the next few lectures, we will look at a few examples of orthobasis expansions that are used in modern signal processing. Cosine transforms The cosine-i transform is an alternative to Fourier series;
More informationLecture 4 Noisy Channel Coding
Lecture 4 Noisy Channel Coding I-Hsiang Wang Department of Electrical Engineering National Taiwan University ihwang@ntu.edu.tw October 9, 2015 1 / 56 I-Hsiang Wang IT Lecture 4 The Channel Coding Problem
More informationE : Lecture 1 Introduction
E85.2607: Lecture 1 Introduction 1 Administrivia 2 DSP review 3 Fun with Matlab E85.2607: Lecture 1 Introduction 2010-01-21 1 / 24 Course overview Advanced Digital Signal Theory Design, analysis, and implementation
More information