Gaussian source Assumptions d = (x-y) 2, given D, find lower bound of I(X;Y)
|
|
- Kenneth Goodman
- 6 years ago
- Views:
Transcription
1 Gaussian source Assumptions d = (x-y) 2, given D, find lower bound of I(X;Y) E{(X-Y) 2 } D<σ 2 I(X;Y ) = H (X) - H (X Y ) = H (X) - H (X -Y Y ) ³ H (X) - H (X -Y ) =.5 log2pes 2 -.5log2peD =.5 log(s 2 / D) Suppose you don t know the characteristics of the source, looks somewhat like Laplacian. Can you comment on R(D). In case of lossless encoding, algo should achieve entropy of source for optimality. Same role is played by R(D) here.
2 Probability Models Uniform distribution Gaussian distribution Laplacian Distribution f X (x) = - 2 x 1 2s e s 2
3 Quantization Quantization: a process of representing a large possibly infinite set of values with a much smaller set. Scalar quantization: a mapping of an input value x into a finite number of output values, y: Codes input
4 Input Codes Output
5 Types of uniform quantizers Midrise (a) quantizers have even number of output levels. Midtread quantizers have odd number of output levels, including zero as one of them
6 Quantization error Since the reconstruction values y i are the midpoints of each interval, the quantization error must lie within the values [ Δ/2, Δ/2]. For a uniformly distributed source, the graph of the quantization error looks like
7 Quantization operation: Let M be the number of reconstruction levels and M boundaries be { b i } i=0 Q(x) = y i if b i-1 < x < b i MSE = s q 2 = ò - ( x - Q(x) ) 2 f X (x)dx M å b i ò = (x - y i ) 2 f X (x)dx i=1 b i-1 Lets represent the quantizer output using fixed length codewords.
8 If there are M levels, the rate is given by R = log 2 M éê For 8 levels we need 3 bit codewords. Quantizer design problem Given f X (x) and M, find decision boundaries b i and reconstruction levels y i, so as to minimize MSE. ùú
9 If we use VLCs to represent yi, then rate Where P(yi) = b i ò b i-1 f X (x) dx M å i=1 R = l i P(y i ) Problem, given distortion constraint s q 2 D * Find decision boundaries, reconstruction levels and binary codes that minimize the rate
10 Uniform Quantization of Uniform Source Input: Uniform [ Xmax, Xmax] Output M level uniform quantizer = 2Xmax/ M M /2 å i=1 id ò (i-1)d s 2 æ q = 2 x - è ç 2i -1 2 D ö ø 2 1 2X max dx = D2 12
11 Consider quantization error instead: q = x Q(x) q [ /2, /2] and is uniformly distributed s 2 q = 1 ò q2 dq = Signal variance E[X 2 ] = X f x dx = = With fixed length codewords, M levels need n bits, M = 2 n
12 SNR db = 20log In terms of n, SNR = 6.02n db For every additional bit we get an increase of 6.02dB.
13 Image compression 1 bit/pixel 2 bit/pixel 3 bit/pixel
14 Uniform Quantization of Nonuniform Sources Example nonuniform source: x [ 100, 100], P(x [ 1, 1]) = 0.95 Problem - Design an 8 level quantizer Previous approach leads to 95% of sample values represented by two numbers: and 12.5 Max quantization error (QE)= 12.5, Min QE = 11.5 Consider an alternative Step = 0.3 Max QE = 98.5, however 95% of the time QE < 0.15 Average distortion would be less in later case.
15 Given M, minimize distortion Distortion as a function of stepsize and minimize the function. σ = 2 x 2i 1 2 To find optimal stepsize, differentiate wrt stepsize and set it equal to 0. Computed using numerical methods Closed form would be very difficult to derive. + 2 x M 1 2 ( ) f x dx f x dx Granular error Overload error
16 Overload and Granular Error
17 Optimum Step Size Using Leibniz rule dσ dδ 2i 1 = 2i 1 x f 2 x dx (M 1) x M 1 f 2 x dx = 0 As we change stepsize, we tradeoff between the two noise profiles.
18 Optimum Δ For the same levels, step-size of the uniform pdf is less than that of Gaussian, which is less than Laplacian. Laplacian has more mass in its tails as compared to Gaussian. So, stepsize should be more to control overload noise.
19 Mismatch Effects Used statistics of the source to determine optimum Δ. However, input may not have the same statistics. Leads to mismatch effects Variance Mismatch; 4 bit Gaussian uniform quantizer with Gaussian input
20 Distribution Mismatch Input pdf does not match the assumed pdf. SNR for different 8 level quantizers. Assume that the sources are uniform, Gaussian, Laplacian, and Gamma. Compute the optimum MSQE step size for uniform quantizer The resulting Δ gets larger from left-to-right
21 Non-uniform quantization For uniform quantizer, decision boundaries are determined by a single parameter Δ. We can certainly reduce quantization errors further if each decision boundaries can be selected freely Boundary selection can be done based on the minimization of error criterion.
22 pdf-optimized Quantization Given pdf, minimize MSE, Find the minimum of function by setting derivative wrt y. s 2 q = ( x - Q(x) ) 2 f X (x)dx y = M å ò - b i ò = (x - y i ) 2 i=1 b i-1 f X (x)dx 2 x y f x dx = 0 y =, b x < b
23 If y j are determined, the b j can be selected as: b = { x y f x dx + x y f x dx = f (b ){ b y - b y } Lloyd-Max quantizer solves iteratively for b j and y j Let us design a mid-rise quantizer, where b 0 =0 and b M/2 = max(input) Problem find {b 1, b 2 b M/2-1 } and reconstruction levels {y 1, y 2, y M/2-1 }
24 Lets put j = 1, we want to find b1 and y1 by y = xf x dx Guess y1 and solve for b1 numerically Then find y 2 = 2b 1 y 1 Find b2 as y = xf x dx f x dx f x dx
25 Find all b s and y s. The accuracy of these values depends on initial guess. We can have a stop criteria as Find b M/2 1 using the previous eqns. Using it find y M/2. Also compute y M/2 from the fact that we know b M/2 and compare them If the difference is less than a threshold stop. Else start the process again with y1.
26 4 bit Laplacian nonuniform quantizer
27 Compander Often the source characteristics vary over time. A usual way to suppress mismatch effect is to use compander c x = 2x if 1 x 1 2x x > 1 2x x < 1
28
29 Expander c x = x 2 if 2 x 2 3x 2 2 x > 2 3x x < 2
30 Equivalent Non-uniform Quantizer
31 If the level of quantizer is large and the input is bounded by x max, it is possible to choose a c(x) such that the SNR of compander is independent to input pdf SNR = 10 log 10 (3M 2 ) 20log 10 α, c x = x sgn x A x = sgn x, 0 x ( ), 1
Scalar and Vector Quantization. National Chiao Tung University Chun-Jen Tsai 11/06/2014
Scalar and Vector Quantization National Chiao Tung University Chun-Jen Tsai 11/06/014 Basic Concept of Quantization Quantization is the process of representing a large, possibly infinite, set of values
More informationMultimedia Communications. Scalar Quantization
Multimedia Communications Scalar Quantization Scalar Quantization In many lossy compression applications we want to represent source outputs using a small number of code words. Process of representing
More informationC.M. Liu Perceptual Signal Processing Lab College of Computer Science National Chiao-Tung University
Quantization C.M. Liu Perceptual Signal Processing Lab College of Computer Science National Chiao-Tung University http://www.csie.nctu.edu.tw/~cmliu/courses/compression/ Office: EC538 (03)5731877 cmliu@cs.nctu.edu.tw
More informationQuantization 2.1 QUANTIZATION AND THE SOURCE ENCODER
2 Quantization After the introduction to image and video compression presented in Chapter 1, we now address several fundamental aspects of image and video compression in the remaining chapters of Section
More informationMultimedia Systems Giorgio Leonardi A.A Lecture 4 -> 6 : Quantization
Multimedia Systems Giorgio Leonardi A.A.2014-2015 Lecture 4 -> 6 : Quantization Overview Course page (D.I.R.): https://disit.dir.unipmn.it/course/view.php?id=639 Consulting: Office hours by appointment:
More informationExample: for source
Nonuniform scalar quantizer References: Sayood Chap. 9, Gersho and Gray, Chap.'s 5 and 6. The basic idea: For a nonuniform source density, put smaller cells and levels where the density is larger, thereby
More informationEE-597 Notes Quantization
EE-597 Notes Quantization Phil Schniter June, 4 Quantization Given a continuous-time and continuous-amplitude signal (t, processing and storage by modern digital hardware requires discretization in both
More informationBeing edited by Prof. Sumana Gupta 1. only symmetric quantizers ie the input and output levels in the 3rd quadrant are negative
Being edited by Prof. Sumana Gupta 1 Quantization This involves representation the sampled data by a finite number of levels based on some criteria such as minimizing of the quantifier distortion. Quantizer
More informationPrinciples of Communications
Principles of Communications Weiyao Lin, PhD Shanghai Jiao Tong University Chapter 4: Analog-to-Digital Conversion Textbook: 7.1 7.4 2010/2011 Meixia Tao @ SJTU 1 Outline Analog signal Sampling Quantization
More informationThe Secrets of Quantization. Nimrod Peleg Update: Sept. 2009
The Secrets of Quantization Nimrod Peleg Update: Sept. 2009 What is Quantization Representation of a large set of elements with a much smaller set is called quantization. The number of elements in the
More informationQuantization. Introduction. Roadmap. Optimal Quantizer Uniform Quantizer Non Uniform Quantizer Rate Distorsion Theory. Source coding.
Roadmap Quantization Optimal Quantizer Uniform Quantizer Non Uniform Quantizer Rate Distorsion Theory Source coding 2 Introduction 4 1 Lossy coding Original source is discrete Lossless coding: bit rate
More informationModule 3 LOSSY IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur
Module 3 LOSSY IMAGE COMPRESSION SYSTEMS Lesson 7 Delta Modulation and DPCM Instructional Objectives At the end of this lesson, the students should be able to: 1. Describe a lossy predictive coding scheme.
More informationCS578- Speech Signal Processing
CS578- Speech Signal Processing Lecture 7: Speech Coding Yannis Stylianou University of Crete, Computer Science Dept., Multimedia Informatics Lab yannis@csd.uoc.gr Univ. of Crete Outline 1 Introduction
More informationCHAPITRE I-5 ETUDE THEORIQUE DE LA ROBUSTESSE DU QUANTIFICATEUR UNIFORME OPTIMUM
CHAPITRE I-5 ETUDE THEORIQUE DE LA ROBUSTESSE DU QUANTIFICATEUR UNIFORME OPTIMUM - 96 - Présentation Après avoir mis au point et validé une méthode d allocation de bits avec un découpage de la DCT en bandes
More informationEE67I Multimedia Communication Systems
EE67I Multimedia Communication Systems Lecture 5: LOSSY COMPRESSION In these schemes, we tradeoff error for bitrate leading to distortion. Lossy compression represents a close approximation of an original
More informationReview of Quantization. Quantization. Bring in Probability Distribution. L-level Quantization. Uniform partition
Review of Quantization UMCP ENEE631 Slides (created by M.Wu 004) Quantization UMCP ENEE631 Slides (created by M.Wu 001/004) L-level Quantization Minimize errors for this lossy process What L values to
More informationRandom Signal Transformations and Quantization
York University Department of Electrical Engineering and Computer Science EECS 4214 Lab #3 Random Signal Transformations and Quantization 1 Purpose In this lab, you will be introduced to transformations
More informationThe information loss in quantization
The information loss in quantization The rough meaning of quantization in the frame of coding is representing numerical quantities with a finite set of symbols. The mapping between numbers, which are normally
More informationPulse-Code Modulation (PCM) :
PCM & DPCM & DM 1 Pulse-Code Modulation (PCM) : In PCM each sample of the signal is quantized to one of the amplitude levels, where B is the number of bits used to represent each sample. The rate from
More informationLec 05 Arithmetic Coding
ECE 5578 Multimedia Communication Lec 05 Arithmetic Coding Zhu Li Dept of CSEE, UMKC web: http://l.web.umkc.edu/lizhu phone: x2346 Z. Li, Multimedia Communciation, 208 p. Outline Lecture 04 ReCap Arithmetic
More information7.1 Sampling and Reconstruction
Haberlesme Sistemlerine Giris (ELE 361) 6 Agustos 2017 TOBB Ekonomi ve Teknoloji Universitesi, Guz 2017-18 Dr. A. Melda Yuksel Turgut & Tolga Girici Lecture Notes Chapter 7 Analog to Digital Conversion
More informationDigital Signal Processing 2/ Advanced Digital Signal Processing Lecture 3, SNR, non-linear Quantisation Gerald Schuller, TU Ilmenau
Digital Signal Processing 2/ Advanced Digital Signal Processing Lecture 3, SNR, non-linear Quantisation Gerald Schuller, TU Ilmenau What is our SNR if we have a sinusoidal signal? What is its pdf? Basically
More informationObjectives of Image Coding
Objectives of Image Coding Representation of an image with acceptable quality, using as small a number of bits as possible Applications: Reduction of channel bandwidth for image transmission Reduction
More informationAn Effective Method for Initialization of Lloyd Max s Algorithm of Optimal Scalar Quantization for Laplacian Source
INFORMATICA, 007, Vol. 18, No., 79 88 79 007 Institute of Mathematics and Informatics, Vilnius An Effective Method for Initialization of Lloyd Max s Algorithm of Optimal Scalar Quantization for Laplacian
More informationDesign of Optimal Quantizers for Distributed Source Coding
Design of Optimal Quantizers for Distributed Source Coding David Rebollo-Monedero, Rui Zhang and Bernd Girod Information Systems Laboratory, Electrical Eng. Dept. Stanford University, Stanford, CA 94305
More informationat Some sort of quantization is necessary to represent continuous signals in digital form
Quantization at Some sort of quantization is necessary to represent continuous signals in digital form x(n 1,n ) x(t 1,tt ) D Sampler Quantizer x q (n 1,nn ) Digitizer (A/D) Quantization is also used for
More informationEE 5345 Biomedical Instrumentation Lecture 12: slides
EE 5345 Biomedical Instrumentation Lecture 1: slides 4-6 Carlos E. Davila, Electrical Engineering Dept. Southern Methodist University slides can be viewed at: http:// www.seas.smu.edu/~cd/ee5345.html EE
More informationVector Quantization. Institut Mines-Telecom. Marco Cagnazzo, MN910 Advanced Compression
Institut Mines-Telecom Vector Quantization Marco Cagnazzo, cagnazzo@telecom-paristech.fr MN910 Advanced Compression 2/66 19.01.18 Institut Mines-Telecom Vector Quantization Outline Gain-shape VQ 3/66 19.01.18
More informationOptimization of Variable-length Code for Data. Compression of memoryless Laplacian source
Optimization of Variable-length Code for Data Compression of memoryless Laplacian source Marko D. Petković, Zoran H. Perić, Aleksandar V. Mosić Abstract In this paper we present the efficient technique
More informationVector Quantization Encoder Decoder Original Form image Minimize distortion Table Channel Image Vectors Look-up (X, X i ) X may be a block of l
Vector Quantization Encoder Decoder Original Image Form image Vectors X Minimize distortion k k Table X^ k Channel d(x, X^ Look-up i ) X may be a block of l m image or X=( r, g, b ), or a block of DCT
More informationE4702 HW#4-5 solutions by Anmo Kim
E70 HW#-5 solutions by Anmo Kim (ak63@columbia.edu). (P3.7) Midtread type uniform quantizer (figure 3.0(a) in Haykin) Gaussian-distributed random variable with zero mean and unit variance is applied to
More informationEE368B Image and Video Compression
EE368B Image and Video Compression Homework Set #2 due Friday, October 20, 2000, 9 a.m. Introduction The Lloyd-Max quantizer is a scalar quantizer which can be seen as a special case of a vector quantizer
More informationCompression methods: the 1 st generation
Compression methods: the 1 st generation 1998-2017 Josef Pelikán CGG MFF UK Praha pepca@cgg.mff.cuni.cz http://cgg.mff.cuni.cz/~pepca/ Still1g 2017 Josef Pelikán, http://cgg.mff.cuni.cz/~pepca 1 / 32 Basic
More informationAudio /Video Signal Processing. Lecture 2, Quantization, SNR Gerald Schuller, TU Ilmenau
Audio /Video Signal Processing Lecture 2, Quantization, SNR Gerald Schuller, TU Ilmenau Quantization Signal to Noise Ratio (SNR). Assume we have a A/D converter with a quantizer with a certain number of
More informationCoding for Discrete Source
EGR 544 Communication Theory 3. Coding for Discrete Sources Z. Aliyazicioglu Electrical and Computer Engineering Department Cal Poly Pomona Coding for Discrete Source Coding Represent source data effectively
More informationDigital Communications III (ECE 154C) Introduction to Coding and Information Theory
Digital Communications III (ECE 154C) Introduction to Coding and Information Theory Tara Javidi These lecture notes were originally developed by late Prof. J. K. Wolf. UC San Diego Spring 2014 1 / 26 Lossy
More informationCh. 8 Math Preliminaries for Lossy Coding. 8.4 Info Theory Revisited
Ch. 8 Math Preliminaries for Lossy Coding 8.4 Info Theory Revisited 1 Info Theory Goals for Lossy Coding Again just as for the lossless case Info Theory provides: Basis for Algorithms & Bounds on Performance
More informationBasic Principles of Video Coding
Basic Principles of Video Coding Introduction Categories of Video Coding Schemes Information Theory Overview of Video Coding Techniques Predictive coding Transform coding Quantization Entropy coding Motion
More informationOverview. Analog capturing device (camera, microphone) PCM encoded or raw signal ( wav, bmp, ) A/D CONVERTER. Compressed bit stream (mp3, jpg, )
Overview Analog capturing device (camera, microphone) Sampling Fine Quantization A/D CONVERTER PCM encoded or raw signal ( wav, bmp, ) Transform Quantizer VLC encoding Compressed bit stream (mp3, jpg,
More informationOn Optimal Coding of Hidden Markov Sources
2014 Data Compression Conference On Optimal Coding of Hidden Markov Sources Mehdi Salehifar, Emrah Akyol, Kumar Viswanatha, and Kenneth Rose Department of Electrical and Computer Engineering University
More information18.2 Continuous Alphabet (discrete-time, memoryless) Channel
0-704: Information Processing and Learning Spring 0 Lecture 8: Gaussian channel, Parallel channels and Rate-distortion theory Lecturer: Aarti Singh Scribe: Danai Koutra Disclaimer: These notes have not
More informationCMPT 365 Multimedia Systems. Final Review - 1
CMPT 365 Multimedia Systems Final Review - 1 Spring 2017 CMPT365 Multimedia Systems 1 Outline Entropy Lossless Compression Shannon-Fano Coding Huffman Coding LZW Coding Arithmetic Coding Lossy Compression
More informationProyecto final de carrera
UPC-ETSETB Proyecto final de carrera A comparison of scalar and vector quantization of wavelet decomposed images Author : Albane Delos Adviser: Luis Torres 2 P a g e Table of contents Table of figures...
More informationE303: Communication Systems
E303: Communication Systems Professor A. Manikas Chair of Communications and Array Processing Imperial College London Principles of PCM Prof. A. Manikas (Imperial College) E303: Principles of PCM v.17
More information1. Probability density function for speech samples. Gamma. Laplacian. 2. Coding paradigms. =(2X max /2 B ) for a B-bit quantizer Δ Δ Δ Δ Δ
Digital Speech Processing Lecture 16 Speech Coding Methods Based on Speech Waveform Representations and Speech Models Adaptive and Differential Coding 1 Speech Waveform Coding-Summary of Part 1 1. Probability
More informationCh. 8 Math Preliminaries for Lossy Coding. 8.5 Rate-Distortion Theory
Ch. 8 Math Preliminaries for Lossy Coding 8.5 Rate-Distortion Theory 1 Introduction Theory provide insight into the trade between Rate & Distortion This theory is needed to answer: What do typical R-D
More informationF O R SOCI AL WORK RESE ARCH
7 TH EUROPE AN CONFERENCE F O R SOCI AL WORK RESE ARCH C h a l l e n g e s i n s o c i a l w o r k r e s e a r c h c o n f l i c t s, b a r r i e r s a n d p o s s i b i l i t i e s i n r e l a t i o n
More informationEE5585 Data Compression February 28, Lecture 11
EE5585 Data Compression February 8, 03 Lecture Instructor: Arya Mazumdar Scribe: Nanwei Yao Rate Distortion Basics When it comes to rate distortion about random variables, there are four important equations
More informationDigital Image Processing Lectures 25 & 26
Lectures 25 & 26, Professor Department of Electrical and Computer Engineering Colorado State University Spring 2015 Area 4: Image Encoding and Compression Goal: To exploit the redundancies in the image
More informationSIGNAL COMPRESSION. 8. Lossy image compression: Principle of embedding
SIGNAL COMPRESSION 8. Lossy image compression: Principle of embedding 8.1 Lossy compression 8.2 Embedded Zerotree Coder 161 8.1 Lossy compression - many degrees of freedom and many viewpoints The fundamental
More informationLecture 20: Quantization and Rate-Distortion
Lecture 20: Quantization and Rate-Distortion Quantization Introduction to rate-distortion theorem Dr. Yao Xie, ECE587, Information Theory, Duke University Approimating continuous signals... Dr. Yao Xie,
More informationStatistical Analysis and Distortion Modeling of MPEG-4 FGS
Statistical Analysis and Distortion Modeling of MPEG-4 FGS Min Dai Electrical Engineering Texas A&M University, TX 77843 Dmitri Loguinov Computer Science Texas A&M University, TX 77843 Hayder Radha Hayder
More informationTransform coding - topics. Principle of block-wise transform coding
Transform coding - topics Principle of block-wise transform coding Properties of orthonormal transforms Discrete cosine transform (DCT) Bit allocation for transform Threshold coding Typical coding artifacts
More informationECE Information theory Final
ECE 776 - Information theory Final Q1 (1 point) We would like to compress a Gaussian source with zero mean and variance 1 We consider two strategies In the first, we quantize with a step size so that the
More informationChapter 9 Fundamental Limits in Information Theory
Chapter 9 Fundamental Limits in Information Theory Information Theory is the fundamental theory behind information manipulation, including data compression and data transmission. 9.1 Introduction o For
More information4. Quantization and Data Compression. ECE 302 Spring 2012 Purdue University, School of ECE Prof. Ilya Pollak
4. Quantization and Data Compression ECE 32 Spring 22 Purdue University, School of ECE Prof. What is data compression? Reducing the file size without compromising the quality of the data stored in the
More informationVID3: Sampling and Quantization
Video Transmission VID3: Sampling and Quantization By Prof. Gregory D. Durgin copyright 2009 all rights reserved Claude E. Shannon (1916-2001) Mathematician and Electrical Engineer Worked for Bell Labs
More informationSCALABLE AUDIO CODING USING WATERMARKING
SCALABLE AUDIO CODING USING WATERMARKING Mahmood Movassagh Peter Kabal Department of Electrical and Computer Engineering McGill University, Montreal, Canada Email: {mahmood.movassagh@mail.mcgill.ca, peter.kabal@mcgill.ca}
More informationPredictive Coding. Prediction
Predictive Coding Prediction Prediction in Images Principle of Differential Pulse Code Modulation (DPCM) DPCM and entropy-constrained scalar quantization DPCM and transmission errors Adaptive intra-interframe
More informationPredictive Coding. Prediction Prediction in Images
Prediction Prediction in Images Predictive Coding Principle of Differential Pulse Code Modulation (DPCM) DPCM and entropy-constrained scalar quantization DPCM and transmission errors Adaptive intra-interframe
More informationEXAMPLE OF SCALAR AND VECTOR QUANTIZATION
EXAMPLE OF SCALAR AD VECTOR QUATIZATIO Source sequence : This could be the output of a highly correlated source. A scalar quantizer: =1, M=4 C 1 = {w 1,w 2,w 3,w 4 } = {-4, -1, 1, 4} = codeboo of quantization
More informationAudio Coding. Fundamentals Quantization Waveform Coding Subband Coding P NCTU/CSIE DSPLAB C.M..LIU
Audio Coding P.1 Fundamentals Quantization Waveform Coding Subband Coding 1. Fundamentals P.2 Introduction Data Redundancy Coding Redundancy Spatial/Temporal Redundancy Perceptual Redundancy Compression
More informationLecture 22: Final Review
Lecture 22: Final Review Nuts and bolts Fundamental questions and limits Tools Practical algorithms Future topics Dr Yao Xie, ECE587, Information Theory, Duke University Basics Dr Yao Xie, ECE587, Information
More informationHomework Set 3 Solutions REVISED EECS 455 Oct. 25, Revisions to solutions to problems 2, 6 and marked with ***
Homework Set 3 Solutions REVISED EECS 455 Oct. 25, 2006 Revisions to solutions to problems 2, 6 and marked with ***. Let U be a continuous random variable with pdf p U (u). Consider an N-point quantizer
More information' 3480 University, Montreal, Quebec H3A 2A7. QUANTIZERS FOR SYMMETRIC GAMMA DISTRIBUTIONS. 3. Uniqueness. 1. Introduction. INRS- TtlCeommunicationr
Proc. IEEE Globecom Conf. (San Diego, CA), pp. 214-218, Nov. 1983 QUANTIZERS FOR SYMMETRIC GAMMA DISTRIBUTIONS Peter Kabd Department of Electrical Engineerinat McGall University INRS- TtlCeommunicationr
More informationClass of waveform coders can be represented in this manner
Digital Speech Processing Lecture 15 Speech Coding Methods Based on Speech Waveform Representations ti and Speech Models Uniform and Non- Uniform Coding Methods 1 Analog-to-Digital Conversion (Sampling
More informationTHE dictionary (Random House) definition of quantization
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 44, NO. 6, OCTOBER 1998 2325 Quantization Robert M. Gray, Fellow, IEEE, and David L. Neuhoff, Fellow, IEEE (Invited Paper) Abstract The history of the theory
More informationDigital Signal Processing
COMP ENG 4TL4: Digital Signal Processing Notes for Lecture #3 Wednesday, September 10, 2003 1.4 Quantization Digital systems can only represent sample amplitudes with a finite set of prescribed values,
More informationMachine Learning and Data Mining. Decision Trees. Prof. Alexander Ihler
+ Machine Learning and Data Mining Decision Trees Prof. Alexander Ihler Decision trees Func-onal form f(x;µ): nested if-then-else statements Discrete features: fully expressive (any func-on) Structure:
More informationJoint Optimum Bitwise Decomposition of any. Memoryless Source to be Sent over a BSC. Ecole Nationale Superieure des Telecommunications URA CNRS 820
Joint Optimum Bitwise Decomposition of any Memoryless Source to be Sent over a BSC Seyed Bahram Zahir Azami, Pierre Duhamel 2 and Olivier Rioul 3 cole Nationale Superieure des Telecommunications URA CNRS
More informationIntroduction p. 1 Compression Techniques p. 3 Lossless Compression p. 4 Lossy Compression p. 5 Measures of Performance p. 5 Modeling and Coding p.
Preface p. xvii Introduction p. 1 Compression Techniques p. 3 Lossless Compression p. 4 Lossy Compression p. 5 Measures of Performance p. 5 Modeling and Coding p. 6 Summary p. 10 Projects and Problems
More informationCommunication Engineering Prof. Surendra Prasad Department of Electrical Engineering Indian Institute of Technology, Delhi
Communication Engineering Prof. Surendra Prasad Department of Electrical Engineering Indian Institute of Technology, Delhi Lecture - 41 Pulse Code Modulation (PCM) So, if you remember we have been talking
More informationPerformance Analysis and Code Optimization of Low Density Parity-Check Codes on Rayleigh Fading Channels
Performance Analysis and Code Optimization of Low Density Parity-Check Codes on Rayleigh Fading Channels Jilei Hou, Paul H. Siegel and Laurence B. Milstein Department of Electrical and Computer Engineering
More informationRemote Estimation Games over Shared Networks
October st, 04 Remote Estimation Games over Shared Networks Marcos Vasconcelos & Nuno Martins marcos@umd.edu Dept. of Electrical and Computer Engineering Institute of Systems Research University of Maryland,
More informationBASICS OF COMPRESSION THEORY
BASICS OF COMPRESSION THEORY Why Compression? Task: storage and transport of multimedia information. E.g.: non-interlaced HDTV: 0x0x0x = Mb/s!! Solutions: Develop technologies for higher bandwidth Find
More informationLecture 3. Mathematical methods in communication I. REMINDER. A. Convex Set. A set R is a convex set iff, x 1,x 2 R, θ, 0 θ 1, θx 1 + θx 2 R, (1)
3- Mathematical methods in communication Lecture 3 Lecturer: Haim Permuter Scribe: Yuval Carmel, Dima Khaykin, Ziv Goldfeld I. REMINDER A. Convex Set A set R is a convex set iff, x,x 2 R, θ, θ, θx + θx
More informationModule 3. Quantization and Coding. Version 2, ECE IIT, Kharagpur
Module Quantization and Coding ersion, ECE IIT, Kharagpur Lesson Logarithmic Pulse Code Modulation (Log PCM) and Companding ersion, ECE IIT, Kharagpur After reading this lesson, you will learn about: Reason
More informationPCM Reference Chapter 12.1, Communication Systems, Carlson. PCM.1
PCM Reference Chapter 1.1, Communication Systems, Carlson. PCM.1 Pulse-code modulation (PCM) Pulse modulations use discrete time samples of analog signals the transmission is composed of analog information
More informationCHAPTER 3. P (B j A i ) P (B j ) =log 2. j=1
CHAPTER 3 Problem 3. : Also : Hence : I(B j ; A i ) = log P (B j A i ) P (B j ) 4 P (B j )= P (B j,a i )= i= 3 P (A i )= P (B j,a i )= j= =log P (B j,a i ) P (B j )P (A i ).3, j=.7, j=.4, j=3.3, i=.7,
More informationQuantization for Distributed Estimation
0 IEEE International Conference on Internet of Things ithings 0), Green Computing and Communications GreenCom 0), and Cyber-Physical-Social Computing CPSCom 0) Quantization for Distributed Estimation uan-yu
More informationT i t l e o f t h e w o r k : L a M a r e a Y o k o h a m a. A r t i s t : M a r i a n o P e n s o t t i ( P l a y w r i g h t, D i r e c t o r )
v e r. E N G O u t l i n e T i t l e o f t h e w o r k : L a M a r e a Y o k o h a m a A r t i s t : M a r i a n o P e n s o t t i ( P l a y w r i g h t, D i r e c t o r ) C o n t e n t s : T h i s w o
More informationImage Coding. Chapter 10. Contents. (Related to Ch. 10 of Lim.) 10.1
Chapter 1 Image Coding Contents Introduction..................................................... 1. Quantization..................................................... 1.3 Scalar quantization...............................................
More informationSeminar: D. Jeon, Energy-efficient Digital Signal Processing Hardware Design Mon Sept 22, 9:30-11:30am in 3316 EECS
EECS 452 Lecture 6 Today: Announcements: Rounding and quantization Analog to digital conversion Lab 3 starts next week Hw3 due on tuesday Project teaming meeting: today 7-9PM, Dow 3150 My new office hours:
More informationChapter 3. Quantization. 3.1 Scalar Quantizers
Chapter 3 Quantization As mentioned in the introduction, two operations are necessary to transform an analog waveform into a digital signal. The first action, sampling, consists of converting a continuous-time
More informationProblem Set III Quantization
Problem Set III Quantization Christopher Tsai Problem #2.1 Lloyd-Max Quantizer To train both the Lloyd-Max Quantizer and our Entropy-Constrained Quantizer, we employ the following training set of images,
More informationarxiv: v1 [cs.it] 21 Feb 2013
q-ary Compressive Sensing arxiv:30.568v [cs.it] Feb 03 Youssef Mroueh,, Lorenzo Rosasco, CBCL, CSAIL, Massachusetts Institute of Technology LCSL, Istituto Italiano di Tecnologia and IIT@MIT lab, Istituto
More informationMARKOV CHAINS A finite state Markov chain is a sequence of discrete cv s from a finite alphabet where is a pmf on and for
MARKOV CHAINS A finite state Markov chain is a sequence S 0,S 1,... of discrete cv s from a finite alphabet S where q 0 (s) is a pmf on S 0 and for n 1, Q(s s ) = Pr(S n =s S n 1 =s ) = Pr(S n =s S n 1
More informationAnalysis of methods for speech signals quantization
INFOTEH-JAHORINA Vol. 14, March 2015. Analysis of methods for speech signals quantization Stefan Stojkov Mihajlo Pupin Institute, University of Belgrade Belgrade, Serbia e-mail: stefan.stojkov@pupin.rs
More informationSolutions to Set #2 Data Compression, Huffman code and AEP
Solutions to Set #2 Data Compression, Huffman code and AEP. Huffman coding. Consider the random variable ( ) x x X = 2 x 3 x 4 x 5 x 6 x 7 0.50 0.26 0. 0.04 0.04 0.03 0.02 (a) Find a binary Huffman code
More informationEncoder Decoder Design for Event-Triggered Feedback Control over Bandlimited Channels
Encoder Decoder Design for Event-Triggered Feedback Control over Bandlimited Channels LEI BAO, MIKAEL SKOGLUND AND KARL HENRIK JOHANSSON IR-EE- 26: Stockholm 26 Signal Processing School of Electrical Engineering
More informationLecture 2: Introduction to Audio, Video & Image Coding Techniques (I) -- Fundaments
Lecture 2: Introduction to Audio, Video & Image Coding Techniques (I) -- Fundaments Dr. Jian Zhang Conjoint Associate Professor NICTA & CSE UNSW COMP9519 Multimedia Systems S2 2006 jzhang@cse.unsw.edu.au
More informationHomework Set #2 Data Compression, Huffman code and AEP
Homework Set #2 Data Compression, Huffman code and AEP 1. Huffman coding. Consider the random variable ( x1 x X = 2 x 3 x 4 x 5 x 6 x 7 0.50 0.26 0.11 0.04 0.04 0.03 0.02 (a Find a binary Huffman code
More informationprobability of k samples out of J fall in R.
Nonparametric Techniques for Density Estimation (DHS Ch. 4) n Introduction n Estimation Procedure n Parzen Window Estimation n Parzen Window Example n K n -Nearest Neighbor Estimation Introduction Suppose
More informationExpectation Maximization
Expectation Maximization Bishop PRML Ch. 9 Alireza Ghane c Ghane/Mori 4 6 8 4 6 8 4 6 8 4 6 8 5 5 5 5 5 5 4 6 8 4 4 6 8 4 5 5 5 5 5 5 µ, Σ) α f Learningscale is slightly Parameters is slightly larger larger
More informationSolutions to Homework Set #1 Sanov s Theorem, Rate distortion
st Semester 00/ Solutions to Homework Set # Sanov s Theorem, Rate distortion. Sanov s theorem: Prove the simple version of Sanov s theorem for the binary random variables, i.e., let X,X,...,X n be a sequence
More informationEE376A - Information Theory Final, Monday March 14th 2016 Solutions. Please start answering each question on a new page of the answer booklet.
EE376A - Information Theory Final, Monday March 14th 216 Solutions Instructions: You have three hours, 3.3PM - 6.3PM The exam has 4 questions, totaling 12 points. Please start answering each question on
More informationLecture Notes 1 Probability and Random Variables. Conditional Probability and Independence. Functions of a Random Variable
Lecture Notes 1 Probability and Random Variables Probability Spaces Conditional Probability and Independence Random Variables Functions of a Random Variable Generation of a Random Variable Jointly Distributed
More informationLecture 2: Introduction to Audio, Video & Image Coding Techniques (I) -- Fundaments. Tutorial 1. Acknowledgement and References for lectures 1 to 5
Lecture : Introduction to Audio, Video & Image Coding Techniques (I) -- Fundaments Dr. Jian Zhang Conjoint Associate Professor NICTA & CSE UNSW COMP959 Multimedia Systems S 006 jzhang@cse.unsw.edu.au Acknowledgement
More informationSimultaneous SDR Optimality via a Joint Matrix Decomp.
Simultaneous SDR Optimality via a Joint Matrix Decomposition Joint work with: Yuval Kochman, MIT Uri Erez, Tel Aviv Uni. May 26, 2011 Model: Source Multicasting over MIMO Channels z 1 H 1 y 1 Rx1 ŝ 1 s
More informationLecture 5 Channel Coding over Continuous Channels
Lecture 5 Channel Coding over Continuous Channels I-Hsiang Wang Department of Electrical Engineering National Taiwan University ihwang@ntu.edu.tw November 14, 2014 1 / 34 I-Hsiang Wang NIT Lecture 5 From
More information