Information Theoretical Analysis of Digital Watermarking. Multimedia Security

Size: px
Start display at page:

Download "Information Theoretical Analysis of Digital Watermarking. Multimedia Security"

Transcription

1 Information Theoretical Analysis of Digital Watermaring Multimedia Security

2 Definitions: X : the output of a source with alphabet X W : a message in a discrete alphabet W={1,2,,M} Assumption : X is a discrete alphabet, follows a discrete distribution S 0,1 : a rv. which indicates whether X will be watermared. P x The variable S is introduced in the model only to provide the possibility of expressing mathematically the existence or nonexistence of a watermar in a simple way. 2

3 K : a secret ey defined on a discrete alphabet. X S=1 : (watermared version) f 1 : x w y (The output of the watermaring function ) f S=0 : X 0 Y (non-watermared version) f 0 : 1 f Y x y f 1 The output of the watermaring function f 1 depends on the value of K, a secret ey which uniquely identifies the copyright owner. 3

4 S X W Y Z K fs P Z Y g ψ q Xˆ Wˆ Ŝ General model of a watermaring system 4

5 The watermared version Y then passes through a noisy channel and is transformed into Z y. This channel models both unintentional distortions suffered by Y and attacs aimed at deleting or corrupting the watermar information. In both cases we assume that the secret ey is not nown, so the noisy channel can be defined by the distribution P Z Y z y which is independent of K. 5

6 x X Finally, Z is processed to obtain a point will be used by the recipient instead of X. There are two tests that can serve to verify the ownership of Z : the watermar detection test q : y 0,1 the watermar decoding test : y w the detection test is used to obtain an estimate which of S (to decide whether Z has been watermared using ) the decoding test is used to obtain an estimate of W. s w 6

7 Imperceptibility : d : x x Let be a perceptually significant distortion. A watermaring system must guarantee that the functions f 0, f1 and g introduce imperceptible alternations with respect to X. E d x, g f x D 0 0 x, g f 1 x, w, D1 E d With expectations taen wrt. X, W, K, (Mean Distortion Consraints) 7

8 8 or (Maximum constraints) K W w X x D w x f g x d X x D x f g x d,,,,,,,,

9 Hiding Information The performance of the watermar decoding process is measured by the probability of error, defined as : P e P r w W P z P, r w 9

10 For each value of K, the space y is partitioned into decision regions D 1,..., D M where M W is the no. of possible hidden messages. Decoding errors are due to the uncertainty about the source output X from which the watermared version was obtained. 10

11 Detecting the Watermar For each value of, the watermar detection test can be mathematically defined as a binary hypothesis test in which we have to decide if Z was x generated by the distribution of or the distribution of f x, w,, where X ~ Px and W is modeled as a random variable. 1 x f 0 11

12 Let Z y q z, 1 be the critical region for the watermar detection test performed with, i.e. the set of point in y where s 1 is decided for that ey. The watermar detection test is completely defined by the sets, K 12

13 13 The performance of the watermar detection test is measured by the probabilities of false alarm and detection, defined as : F P P D r r D r r F s Z P P s s P P s Z P P s s P P 1,

14 Suppose there is no distortion during distribution, so Z=Y optimizing the performance of the watermar detection test in terms of PF and PD is in a way equivalent to maximizing the Kullbac- Laibler distance between distributions : P Y s 1, and PY s 0 The maximum achievable distance is limited by the perceptual distortion constraint and entropy of the source. 14

15 The probability of collision between eys and : the probability of deciding s 1 in the watermar detection test for certain ey K 1 when Z has been watermared using a different ey K 2. In the context of copyright protection, this probability should be constrained below a maximum allowed value for all pairs ( K 1, K 2 ) since otherwise the author in possession of K 1 could claim authorship of information watermared by the author who owns. K 2 K 1 K 2 15

16 This constraint imposes a limit to the cardinality of the ey space since the minimum achievable maximum probability of collision between eys increase with the number of eys for fixed P and P. F D 16

17 Attacs In the following discussion we will assume that the attacer has unlimited computation power and that the algorithm for watermaring, detection and decoding are public. The security of the watermaring system relies exclusively on the secret ey K of the copyright owner. 17

18 The Elimination Attac Alternate a watermared source output Y to obtain a negative result s 0 in the watermar detection test for the secret ey used by the legitimate owner. The alteration made by the attacer should not be perceptible, since the resulting output Z will be used as a substitute for the watermared source output Y. 18

19 This constraint can be expressed in mathematical form as an average distortion constraint Ed Z, Y DE or as a maximum distortion constraint dz, Y DE, Z, Y, where d(.,.) is a distortion function and D E is the maximum distortion allowed by the attacer. 19

20 The Elimination Attac can be represented by a game-theoretic model : Given a certain watermared source output Y, the attacer will choose the point Z y, subject to the distortion constraint, which maximizes his probability of success. 20

21 Under a maximum distortion constraint, this maximum probability of success for a given Y can be expressed as P E Y Z : d max Z, Y D E K Y 1 qz, K After averaging out over y, the average probability of success in the elimination attac is max P E P Y P K Y 1 q Z, K Z : d Z, Y D Y E P 21

22 We can model the transformation made by the attacer as a channel with conditioned pdf P Z. Y Then the optimal elimination strategy can be seen as a worst-case channel P Z in the sense that it Y minimizes the P D for given critical regions and watermaring function. f 1 Note that the attacer is limited to those channels which satisfy the average distortion constraint. 22

23 The minimum achievable function of. D E is a non-increasing The optimum watermaring strategy consists in choosing the watermaring function f 1 and the critical regions maximizing the minimum P D achievable by the attacer through the choice of a channel P Z Y.Hence, the design of the watermaring system is a robust hypothesis testing problem. P D 23

24 The Corruption Attac The attacer is not interested in eliminating the watermar, but increasing the probability of error in the watermar decoding process. 24

25 Cryptographic Security The securing level of the system can be measured by the uncertainty about the ey given a watermared source output Y. Using an information-theoretical terminology, this uncertainty is the conditioned entropy HK Y, also called equivocation. 25

26 Size of Key Space A minimum cardinality of the ey space K is a necessary condition for specifying the equivocation HK Y. Increasing the equivocation helps in increasing the robustness against elimination attacs. However, increasing the number of available eys also increases the probability of collision among eys. Therefore, if we specify a maximum allowable probability of collision, this constraint will impose a limit on the maximum number of eys. 26

27 Summary Decoding of hidden information is affected by uncertainty due to the source output (not available at the receiver), distortion and attacs. We can thin that there is a channel between W and Z which can be characterized by a certain capacity. Watermaring and watermar detection under a constrained maximum probability of collision between eys can be seen as an application of identification via channels, with additional constraints derived from the limited admissible perceptual distortion in the watermaring process. The combination of watermar detection and data hiding can be related to the theory of identification plus transmission codes. 27

Spread Spectrum Watermarking. Information Technologies for IPR Protection

Spread Spectrum Watermarking. Information Technologies for IPR Protection Spread Spectrum Watermarking Information Technologies for IPR Protection Spread Spectrum Communication A Narrow-band signal is transmitted over a much larger bandwidth such that the signal energy presented

More information

The Method of Types and Its Application to Information Hiding

The Method of Types and Its Application to Information Hiding The Method of Types and Its Application to Information Hiding Pierre Moulin University of Illinois at Urbana-Champaign www.ifp.uiuc.edu/ moulin/talks/eusipco05-slides.pdf EUSIPCO Antalya, September 7,

More information

and its Extension to Authenticity

and its Extension to Authenticity EWSCS 06 almse, Estonia 5-10 March 2006 Lecture 1: Shannon s Theory of Secrecy and its Extension to Authenticity James L. Massey rof.-em. ETH Zürich, Adjunct rof., Lund Univ., Sweden, and Tech. Univ. of

More information

Audio Zero-Watermarking Based on Discrete Hartley Transform and Non-Negative Matrix Factorization

Audio Zero-Watermarking Based on Discrete Hartley Transform and Non-Negative Matrix Factorization IJCSNS International Journal of Computer Science and Networ Security, VOL7 No8, August 207 Audio Zero-Watermaring Based on Discrete Hartley Transform and Non-Negative Matrix Factorization Pranab Kumar

More information

Quantum Wireless Sensor Networks

Quantum Wireless Sensor Networks Quantum Wireless Sensor Networks School of Computing Queen s University Canada ntional Computation Vienna, August 2008 Main Result Quantum cryptography can solve the problem of security in sensor networks.

More information

On the Bayes Risk in Information-Hiding Protocols

On the Bayes Risk in Information-Hiding Protocols On the Bayes Risk in Information-Hiding Protocols Konstantinos Chatzikokolakis Catuscia Palamidessi INRIA and LIX, École Polytechnique Palaiseau, France {kostas,catuscia}@lix.polytechnique.fr Prakash Panangaden

More information

Exercise 1. = P(y a 1)P(a 1 )

Exercise 1. = P(y a 1)P(a 1 ) Chapter 7 Channel Capacity Exercise 1 A source produces independent, equally probable symbols from an alphabet {a 1, a 2 } at a rate of one symbol every 3 seconds. These symbols are transmitted over a

More information

CS6304 / Analog and Digital Communication UNIT IV - SOURCE AND ERROR CONTROL CODING PART A 1. What is the use of error control coding? The main use of error control coding is to reduce the overall probability

More information

University of Siena. Multimedia Security. Watermark extraction. Mauro Barni University of Siena. M. Barni, University of Siena

University of Siena. Multimedia Security. Watermark extraction. Mauro Barni University of Siena. M. Barni, University of Siena Multimedia Security Mauro Barni University of Siena : summary Optimum decoding/detection Additive SS watermarks Decoding/detection of QIM watermarks The dilemma of de-synchronization attacks Geometric

More information

SHARED INFORMATION. Prakash Narayan with. Imre Csiszár, Sirin Nitinawarat, Himanshu Tyagi, Shun Watanabe

SHARED INFORMATION. Prakash Narayan with. Imre Csiszár, Sirin Nitinawarat, Himanshu Tyagi, Shun Watanabe SHARED INFORMATION Prakash Narayan with Imre Csiszár, Sirin Nitinawarat, Himanshu Tyagi, Shun Watanabe 2/40 Acknowledgement Praneeth Boda Himanshu Tyagi Shun Watanabe 3/40 Outline Two-terminal model: Mutual

More information

Lecture 7. Union bound for reducing M-ary to binary hypothesis testing

Lecture 7. Union bound for reducing M-ary to binary hypothesis testing Lecture 7 Agenda for the lecture M-ary hypothesis testing and the MAP rule Union bound for reducing M-ary to binary hypothesis testing Introduction of the channel coding problem 7.1 M-ary hypothesis testing

More information

NOISE can sometimes improve nonlinear signal processing

NOISE can sometimes improve nonlinear signal processing 488 IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 59, NO. 2, FEBRUARY 2011 Noise Benefits in Quantizer-Array Correlation Detection and Watermark Decoding Ashok Patel, Member, IEEE, and Bart Kosko, Fellow,

More information

Communication Theory II

Communication Theory II Communication Theory II Lecture 15: Information Theory (cont d) Ahmed Elnakib, PhD Assistant Professor, Mansoura University, Egypt March 29 th, 2015 1 Example: Channel Capacity of BSC o Let then: o For

More information

Frans M.J. Willems. Authentication Based on Secret-Key Generation. Frans M.J. Willems. (joint work w. Tanya Ignatenko)

Frans M.J. Willems. Authentication Based on Secret-Key Generation. Frans M.J. Willems. (joint work w. Tanya Ignatenko) Eindhoven University of Technology IEEE EURASIP Spain Seminar on Signal Processing, Communication and Information Theory, Universidad Carlos III de Madrid, December 11, 2014 : Secret-Based Authentication

More information

Digital Communications III (ECE 154C) Introduction to Coding and Information Theory

Digital Communications III (ECE 154C) Introduction to Coding and Information Theory Digital Communications III (ECE 154C) Introduction to Coding and Information Theory Tara Javidi These lecture notes were originally developed by late Prof. J. K. Wolf. UC San Diego Spring 2014 1 / 8 I

More information

Achieving Shannon Capacity Region as Secrecy Rate Region in a Multiple Access Wiretap Channel

Achieving Shannon Capacity Region as Secrecy Rate Region in a Multiple Access Wiretap Channel Achieving Shannon Capacity Region as Secrecy Rate Region in a Multiple Access Wiretap Channel Shahid Mehraj Shah and Vinod Sharma Department of Electrical Communication Engineering, Indian Institute of

More information

ELEC546 Review of Information Theory

ELEC546 Review of Information Theory ELEC546 Review of Information Theory Vincent Lau 1/1/004 1 Review of Information Theory Entropy: Measure of uncertainty of a random variable X. The entropy of X, H(X), is given by: If X is a discrete random

More information

Outline. Computer Science 418. Number of Keys in the Sum. More on Perfect Secrecy, One-Time Pad, Entropy. Mike Jacobson. Week 3

Outline. Computer Science 418. Number of Keys in the Sum. More on Perfect Secrecy, One-Time Pad, Entropy. Mike Jacobson. Week 3 Outline Computer Science 48 More on Perfect Secrecy, One-Time Pad, Mike Jacobson Department of Computer Science University of Calgary Week 3 2 3 Mike Jacobson (University of Calgary) Computer Science 48

More information

Bioinformatics: Biology X

Bioinformatics: Biology X Bud Mishra Room 1002, 715 Broadway, Courant Institute, NYU, New York, USA Model Building/Checking, Reverse Engineering, Causality Outline 1 Bayesian Interpretation of Probabilities 2 Where (or of what)

More information

Multimedia Communications. Scalar Quantization

Multimedia Communications. Scalar Quantization Multimedia Communications Scalar Quantization Scalar Quantization In many lossy compression applications we want to represent source outputs using a small number of code words. Process of representing

More information

AN INTRODUCTION TO SECRECY CAPACITY. 1. Overview

AN INTRODUCTION TO SECRECY CAPACITY. 1. Overview AN INTRODUCTION TO SECRECY CAPACITY BRIAN DUNN. Overview This paper introduces the reader to several information theoretic aspects of covert communications. In particular, it discusses fundamental limits

More information

Probability review. September 11, Stoch. Systems Analysis Introduction 1

Probability review. September 11, Stoch. Systems Analysis Introduction 1 Probability review Alejandro Ribeiro Dept. of Electrical and Systems Engineering University of Pennsylvania aribeiro@seas.upenn.edu http://www.seas.upenn.edu/users/~aribeiro/ September 11, 2015 Stoch.

More information

Information Hiding and Covert Communication

Information Hiding and Covert Communication Information Hiding and Covert Communication Andrew Ker adk @ comlab.ox.ac.uk Royal Society University Research Fellow Oxford University Computing Laboratory Foundations of Security Analysis and Design

More information

Secret Key Agreement Using Asymmetry in Channel State Knowledge

Secret Key Agreement Using Asymmetry in Channel State Knowledge Secret Key Agreement Using Asymmetry in Channel State Knowledge Ashish Khisti Deutsche Telekom Inc. R&D Lab USA Los Altos, CA, 94040 Email: ashish.khisti@telekom.com Suhas Diggavi LICOS, EFL Lausanne,

More information

Introduction to Information Theory. B. Škorić, Physical Aspects of Digital Security, Chapter 2

Introduction to Information Theory. B. Škorić, Physical Aspects of Digital Security, Chapter 2 Introduction to Information Theory B. Škorić, Physical Aspects of Digital Security, Chapter 2 1 Information theory What is it? - formal way of counting information bits Why do we need it? - often used

More information

Shannon s Noisy-Channel Coding Theorem

Shannon s Noisy-Channel Coding Theorem Shannon s Noisy-Channel Coding Theorem Lucas Slot Sebastian Zur February 13, 2015 Lucas Slot, Sebastian Zur Shannon s Noisy-Channel Coding Theorem February 13, 2015 1 / 29 Outline 1 Definitions and Terminology

More information

Channel capacity. Outline : 1. Source entropy 2. Discrete memoryless channel 3. Mutual information 4. Channel capacity 5.

Channel capacity. Outline : 1. Source entropy 2. Discrete memoryless channel 3. Mutual information 4. Channel capacity 5. Channel capacity Outline : 1. Source entropy 2. Discrete memoryless channel 3. Mutual information 4. Channel capacity 5. Exercices Exercise session 11 : Channel capacity 1 1. Source entropy Given X a memoryless

More information

Information and Entropy

Information and Entropy Information and Entropy Shannon s Separation Principle Source Coding Principles Entropy Variable Length Codes Huffman Codes Joint Sources Arithmetic Codes Adaptive Codes Thomas Wiegand: Digital Image Communication

More information

BASICS OF DETECTION AND ESTIMATION THEORY

BASICS OF DETECTION AND ESTIMATION THEORY BASICS OF DETECTION AND ESTIMATION THEORY 83050E/158 In this chapter we discuss how the transmitted symbols are detected optimally from a noisy received signal (observation). Based on these results, optimal

More information

LECTURE 15. Last time: Feedback channel: setting up the problem. Lecture outline. Joint source and channel coding theorem

LECTURE 15. Last time: Feedback channel: setting up the problem. Lecture outline. Joint source and channel coding theorem LECTURE 15 Last time: Feedback channel: setting up the problem Perfect feedback Feedback capacity Data compression Lecture outline Joint source and channel coding theorem Converse Robustness Brain teaser

More information

Quantum Entanglement and Cryptography. Deepthi Gopal, Caltech

Quantum Entanglement and Cryptography. Deepthi Gopal, Caltech + Quantum Entanglement and Cryptography Deepthi Gopal, Caltech + Cryptography Concisely: to make information unreadable by anyone other than the intended recipient. The sender of a message scrambles/encrypts

More information

Lecture 12. Block Diagram

Lecture 12. Block Diagram Lecture 12 Goals Be able to encode using a linear block code Be able to decode a linear block code received over a binary symmetric channel or an additive white Gaussian channel XII-1 Block Diagram Data

More information

Coding for Computing. ASPITRG, Drexel University. Jie Ren 2012/11/14

Coding for Computing. ASPITRG, Drexel University. Jie Ren 2012/11/14 Coding for Computing ASPITRG, Drexel University Jie Ren 2012/11/14 Outline Background and definitions Main Results Examples and Analysis Proofs Background and definitions Problem Setup Graph Entropy Conditional

More information

Channel Coding I. Exercises SS 2017

Channel Coding I. Exercises SS 2017 Channel Coding I Exercises SS 2017 Lecturer: Dirk Wübben Tutor: Shayan Hassanpour NW1, Room N 2420, Tel.: 0421/218-62387 E-mail: {wuebben, hassanpour}@ant.uni-bremen.de Universität Bremen, FB1 Institut

More information

INFORMATION PROCESSING ABILITY OF BINARY DETECTORS AND BLOCK DECODERS. Michael A. Lexa and Don H. Johnson

INFORMATION PROCESSING ABILITY OF BINARY DETECTORS AND BLOCK DECODERS. Michael A. Lexa and Don H. Johnson INFORMATION PROCESSING ABILITY OF BINARY DETECTORS AND BLOCK DECODERS Michael A. Lexa and Don H. Johnson Rice University Department of Electrical and Computer Engineering Houston, TX 775-892 amlexa@rice.edu,

More information

CSCI 2570 Introduction to Nanocomputing

CSCI 2570 Introduction to Nanocomputing CSCI 2570 Introduction to Nanocomputing Information Theory John E Savage What is Information Theory Introduced by Claude Shannon. See Wikipedia Two foci: a) data compression and b) reliable communication

More information

Cut-Set Bound and Dependence Balance Bound

Cut-Set Bound and Dependence Balance Bound Cut-Set Bound and Dependence Balance Bound Lei Xiao lxiao@nd.edu 1 Date: 4 October, 2006 Reading: Elements of information theory by Cover and Thomas [1, Section 14.10], and the paper by Hekstra and Willems

More information

1 Introduction to information theory

1 Introduction to information theory 1 Introduction to information theory 1.1 Introduction In this chapter we present some of the basic concepts of information theory. The situations we have in mind involve the exchange of information through

More information

Lecture 14 February 28

Lecture 14 February 28 EE/Stats 376A: Information Theory Winter 07 Lecture 4 February 8 Lecturer: David Tse Scribe: Sagnik M, Vivek B 4 Outline Gaussian channel and capacity Information measures for continuous random variables

More information

On the Bayes Risk in Information-Hiding Protocols

On the Bayes Risk in Information-Hiding Protocols On the Bayes Risk in Information-Hiding Protocols Konstantinos Chatzikokolakis Catuscia Palamidessi INRIA and LIX, École Polytechnique Palaiseau, France {kostas,catuscia}@lix.polytechnique.fr Prakash Panangaden

More information

Lecture 4 Noisy Channel Coding

Lecture 4 Noisy Channel Coding Lecture 4 Noisy Channel Coding I-Hsiang Wang Department of Electrical Engineering National Taiwan University ihwang@ntu.edu.tw October 9, 2015 1 / 56 I-Hsiang Wang IT Lecture 4 The Channel Coding Problem

More information

Decentralized Detection In Wireless Sensor Networks

Decentralized Detection In Wireless Sensor Networks Decentralized Detection In Wireless Sensor Networks Milad Kharratzadeh Department of Electrical & Computer Engineering McGill University Montreal, Canada April 2011 Statistical Detection and Estimation

More information

Capacity of the Discrete Memoryless Energy Harvesting Channel with Side Information

Capacity of the Discrete Memoryless Energy Harvesting Channel with Side Information 204 IEEE International Symposium on Information Theory Capacity of the Discrete Memoryless Energy Harvesting Channel with Side Information Omur Ozel, Kaya Tutuncuoglu 2, Sennur Ulukus, and Aylin Yener

More information

ECE Information theory Final

ECE Information theory Final ECE 776 - Information theory Final Q1 (1 point) We would like to compress a Gaussian source with zero mean and variance 1 We consider two strategies In the first, we quantize with a step size so that the

More information

Wavelet Packet Based Digital Image Watermarking

Wavelet Packet Based Digital Image Watermarking Wavelet Packet Based Digital Image ing A.Adhipathi Reddy, B.N.Chatterji Department of Electronics and Electrical Communication Engg. Indian Institute of Technology, Kharagpur 72 32 {aar, bnc}@ece.iitkgp.ernet.in

More information

Shannon s noisy-channel theorem

Shannon s noisy-channel theorem Shannon s noisy-channel theorem Information theory Amon Elders Korteweg de Vries Institute for Mathematics University of Amsterdam. Tuesday, 26th of Januari Amon Elders (Korteweg de Vries Institute for

More information

Compression and Coding

Compression and Coding Compression and Coding Theory and Applications Part 1: Fundamentals Gloria Menegaz 1 Transmitter (Encoder) What is the problem? Receiver (Decoder) Transformation information unit Channel Ordering (significance)

More information

X 1 : X Table 1: Y = X X 2

X 1 : X Table 1: Y = X X 2 ECE 534: Elements of Information Theory, Fall 200 Homework 3 Solutions (ALL DUE to Kenneth S. Palacio Baus) December, 200. Problem 5.20. Multiple access (a) Find the capacity region for the multiple-access

More information

6.02 Fall 2011 Lecture #9

6.02 Fall 2011 Lecture #9 6.02 Fall 2011 Lecture #9 Claude E. Shannon Mutual information Channel capacity Transmission at rates up to channel capacity, and with asymptotically zero error 6.02 Fall 2011 Lecture 9, Slide #1 First

More information

A Framework for Optimizing Nonlinear Collusion Attacks on Fingerprinting Systems

A Framework for Optimizing Nonlinear Collusion Attacks on Fingerprinting Systems A Framework for Optimizing onlinear Collusion Attacks on Fingerprinting Systems egar iyavash Department of Electrical and Computer Engineering University of Illinois at Urbana-Champaign Coordinated Science

More information

1. INTRODUCTION. Yen-Chung Chiu. Wen-Hsiang Tsai.

1. INTRODUCTION. Yen-Chung Chiu. Wen-Hsiang Tsai. Copyright Protection by Watermarking for Color Images against Rotation and Scaling Attacks Using Coding and Synchronization of Peak Locations in DFT Domain Yen-Chung Chiu Department of Computer and Information

More information

MAHALAKSHMI ENGINEERING COLLEGE-TRICHY QUESTION BANK UNIT V PART-A. 1. What is binary symmetric channel (AUC DEC 2006)

MAHALAKSHMI ENGINEERING COLLEGE-TRICHY QUESTION BANK UNIT V PART-A. 1. What is binary symmetric channel (AUC DEC 2006) MAHALAKSHMI ENGINEERING COLLEGE-TRICHY QUESTION BANK SATELLITE COMMUNICATION DEPT./SEM.:ECE/VIII UNIT V PART-A 1. What is binary symmetric channel (AUC DEC 2006) 2. Define information rate? (AUC DEC 2007)

More information

Entropies & Information Theory

Entropies & Information Theory Entropies & Information Theory LECTURE I Nilanjana Datta University of Cambridge,U.K. See lecture notes on: http://www.qi.damtp.cam.ac.uk/node/223 Quantum Information Theory Born out of Classical Information

More information

Computational Systems Biology: Biology X

Computational Systems Biology: Biology X Bud Mishra Room 1002, 715 Broadway, Courant Institute, NYU, New York, USA L#8:(November-08-2010) Cancer and Signals Outline 1 Bayesian Interpretation of Probabilities Information Theory Outline Bayesian

More information

Revision of Lecture 5

Revision of Lecture 5 Revision of Lecture 5 Information transferring across channels Channel characteristics and binary symmetric channel Average mutual information Average mutual information tells us what happens to information

More information

Module 1. Introduction to Digital Communications and Information Theory. Version 2 ECE IIT, Kharagpur

Module 1. Introduction to Digital Communications and Information Theory. Version 2 ECE IIT, Kharagpur Module ntroduction to Digital Communications and nformation Theory Lesson 3 nformation Theoretic Approach to Digital Communications After reading this lesson, you will learn about Scope of nformation Theory

More information

Lecture 4 Channel Coding

Lecture 4 Channel Coding Capacity and the Weak Converse Lecture 4 Coding I-Hsiang Wang Department of Electrical Engineering National Taiwan University ihwang@ntu.edu.tw October 15, 2014 1 / 16 I-Hsiang Wang NIT Lecture 4 Capacity

More information

Coding for Discrete Source

Coding for Discrete Source EGR 544 Communication Theory 3. Coding for Discrete Sources Z. Aliyazicioglu Electrical and Computer Engineering Department Cal Poly Pomona Coding for Discrete Source Coding Represent source data effectively

More information

MAHALAKSHMI ENGINEERING COLLEGE QUESTION BANK. SUBJECT CODE / Name: EC2252 COMMUNICATION THEORY UNIT-V INFORMATION THEORY PART-A

MAHALAKSHMI ENGINEERING COLLEGE QUESTION BANK. SUBJECT CODE / Name: EC2252 COMMUNICATION THEORY UNIT-V INFORMATION THEORY PART-A MAHALAKSHMI ENGINEERING COLLEGE QUESTION BANK DEPARTMENT: ECE SEMESTER: IV SUBJECT CODE / Name: EC2252 COMMUNICATION THEORY UNIT-V INFORMATION THEORY PART-A 1. What is binary symmetric channel (AUC DEC

More information

Information Theoretic Security and Privacy of Information Systems. Edited by Holger Boche, Ashish Khisti, H. Vincent Poor, and Rafael F.

Information Theoretic Security and Privacy of Information Systems. Edited by Holger Boche, Ashish Khisti, H. Vincent Poor, and Rafael F. Information Theoretic Security and Privacy of Information Systems Edited by Holger Boche, Ashish Khisti, H. Vincent Poor, and Rafael F. Schaefer 1 Secure source coding Paul Cuff and Curt Schieler Abstract

More information

Noisy channel communication

Noisy channel communication Information Theory http://www.inf.ed.ac.uk/teaching/courses/it/ Week 6 Communication channels and Information Some notes on the noisy channel setup: Iain Murray, 2012 School of Informatics, University

More information

Appendix B Information theory from first principles

Appendix B Information theory from first principles Appendix B Information theory from first principles This appendix discusses the information theory behind the capacity expressions used in the book. Section 8.3.4 is the only part of the book that supposes

More information

Information Theory. Coding and Information Theory. Information Theory Textbooks. Entropy

Information Theory. Coding and Information Theory. Information Theory Textbooks. Entropy Coding and Information Theory Chris Williams, School of Informatics, University of Edinburgh Overview What is information theory? Entropy Coding Information Theory Shannon (1948): Information theory is

More information

SHARED INFORMATION. Prakash Narayan with. Imre Csiszár, Sirin Nitinawarat, Himanshu Tyagi, Shun Watanabe

SHARED INFORMATION. Prakash Narayan with. Imre Csiszár, Sirin Nitinawarat, Himanshu Tyagi, Shun Watanabe SHARED INFORMATION Prakash Narayan with Imre Csiszár, Sirin Nitinawarat, Himanshu Tyagi, Shun Watanabe 2/41 Outline Two-terminal model: Mutual information Operational meaning in: Channel coding: channel

More information

Lecture 1. Introduction

Lecture 1. Introduction Lecture 1. Introduction What is the course about? Logistics Questionnaire Dr. Yao Xie, ECE587, Information Theory, Duke University What is information? Dr. Yao Xie, ECE587, Information Theory, Duke University

More information

Information Theory - Entropy. Figure 3

Information Theory - Entropy. Figure 3 Concept of Information Information Theory - Entropy Figure 3 A typical binary coded digital communication system is shown in Figure 3. What is involved in the transmission of information? - The system

More information

Quiz 2 Date: Monday, November 21, 2016

Quiz 2 Date: Monday, November 21, 2016 10-704 Information Processing and Learning Fall 2016 Quiz 2 Date: Monday, November 21, 2016 Name: Andrew ID: Department: Guidelines: 1. PLEASE DO NOT TURN THIS PAGE UNTIL INSTRUCTED. 2. Write your name,

More information

Reed-Solomon code. P(n + 2k)

Reed-Solomon code. P(n + 2k) Reed-Solomon code. Problem: Communicate n packets m 1,...,m n on noisy channel that corrupts k packets. Reed-Solomon Code: 1. Make a polynomial, P(x) of degree n 1, that encodes message: coefficients,

More information

Bounding the number of affine roots

Bounding the number of affine roots with applications in reliable and secure communication Inaugural Lecture, Aalborg University, August 11110, 11111100000 with applications in reliable and secure communication Polynomials: F (X ) = 2X 2

More information

Upper Bounds on the Capacity of Binary Intermittent Communication

Upper Bounds on the Capacity of Binary Intermittent Communication Upper Bounds on the Capacity of Binary Intermittent Communication Mostafa Khoshnevisan and J. Nicholas Laneman Department of Electrical Engineering University of Notre Dame Notre Dame, Indiana 46556 Email:{mhoshne,

More information

Lecture 7 Introduction to Statistical Decision Theory

Lecture 7 Introduction to Statistical Decision Theory Lecture 7 Introduction to Statistical Decision Theory I-Hsiang Wang Department of Electrical Engineering National Taiwan University ihwang@ntu.edu.tw December 20, 2016 1 / 55 I-Hsiang Wang IT Lecture 7

More information

Noisy-Channel Coding

Noisy-Channel Coding Copyright Cambridge University Press 2003. On-screen viewing permitted. Printing not permitted. http://www.cambridge.org/05264298 Part II Noisy-Channel Coding Copyright Cambridge University Press 2003.

More information

ELECTRONICS & COMMUNICATIONS DIGITAL COMMUNICATIONS

ELECTRONICS & COMMUNICATIONS DIGITAL COMMUNICATIONS EC 32 (CR) Total No. of Questions :09] [Total No. of Pages : 02 III/IV B.Tech. DEGREE EXAMINATIONS, APRIL/MAY- 207 Second Semester ELECTRONICS & COMMUNICATIONS DIGITAL COMMUNICATIONS Time: Three Hours

More information

MPRI Course on Concurrency. Lecture 14. Application of probabilistic process calculi to security

MPRI Course on Concurrency. Lecture 14. Application of probabilistic process calculi to security MPRI Course on Concurrency Lecture 14 Application of probabilistic process calculi to security Catuscia Palamidessi LIX, Ecole Polytechnique kostas@lix.polytechnique.fr www.lix.polytechnique.fr/~catuscia

More information

STEGOSYSTEMS BASED ON NOISY CHANNELS

STEGOSYSTEMS BASED ON NOISY CHANNELS International Journal of Computer Science and Applications c Technomathematics Research Foundation Vol. 8 No. 1, pp. 1-13, 011 STEGOSYSTEMS BASED ON NOISY CHANNELS VALERY KORZHIK State University of Telecommunications,

More information

On Sequential Watermark Detection

On Sequential Watermark Detection On Sequential Watermark Detection R. Chandramouli and Nasir D. Memon Abstract This paper introduces a new watermark detection paradigm based on sequential detection. It is shown that this framework leads

More information

ECE Advanced Communication Theory, Spring 2009 Homework #1 (INCOMPLETE)

ECE Advanced Communication Theory, Spring 2009 Homework #1 (INCOMPLETE) ECE 74 - Advanced Communication Theory, Spring 2009 Homework #1 (INCOMPLETE) 1. A Huffman code finds the optimal codeword to assign to a given block of source symbols. (a) Show that cannot be a Huffman

More information

Network Coding and Schubert Varieties over Finite Fields

Network Coding and Schubert Varieties over Finite Fields Network Coding and Schubert Varieties over Finite Fields Anna-Lena Horlemann-Trautmann Algorithmics Laboratory, EPFL, Schweiz October 12th, 2016 University of Kentucky What is this talk about? 1 / 31 Overview

More information

Chapter 9 Fundamental Limits in Information Theory

Chapter 9 Fundamental Limits in Information Theory Chapter 9 Fundamental Limits in Information Theory Information Theory is the fundamental theory behind information manipulation, including data compression and data transmission. 9.1 Introduction o For

More information

Example: sending one bit of information across noisy channel. Effects of the noise: flip the bit with probability p.

Example: sending one bit of information across noisy channel. Effects of the noise: flip the bit with probability p. Lecture 20 Page 1 Lecture 20 Quantum error correction Classical error correction Modern computers: failure rate is below one error in 10 17 operations Data transmission and storage (file transfers, cell

More information

The peculiarities of the model: - it is allowed to use by attacker only noisy version of SG C w (n), - CO can be known exactly for attacker.

The peculiarities of the model: - it is allowed to use by attacker only noisy version of SG C w (n), - CO can be known exactly for attacker. Lecture 6. SG based on noisy channels [4]. CO Detection of SG The peculiarities of the model: - it is alloed to use by attacker only noisy version of SG C (n), - CO can be knon exactly for attacker. Practical

More information

Data Detection for Controlled ISI. h(nt) = 1 for n=0,1 and zero otherwise.

Data Detection for Controlled ISI. h(nt) = 1 for n=0,1 and zero otherwise. Data Detection for Controlled ISI *Symbol by symbol suboptimum detection For the duobinary signal pulse h(nt) = 1 for n=0,1 and zero otherwise. The samples at the output of the receiving filter(demodulator)

More information

Lecture 5 Channel Coding over Continuous Channels

Lecture 5 Channel Coding over Continuous Channels Lecture 5 Channel Coding over Continuous Channels I-Hsiang Wang Department of Electrical Engineering National Taiwan University ihwang@ntu.edu.tw November 14, 2014 1 / 34 I-Hsiang Wang NIT Lecture 5 From

More information

One Lesson of Information Theory

One Lesson of Information Theory Institut für One Lesson of Information Theory Prof. Dr.-Ing. Volker Kühn Institute of Communications Engineering University of Rostock, Germany Email: volker.kuehn@uni-rostock.de http://www.int.uni-rostock.de/

More information

Introduction to Information Theory. Uncertainty. Entropy. Surprisal. Joint entropy. Conditional entropy. Mutual information.

Introduction to Information Theory. Uncertainty. Entropy. Surprisal. Joint entropy. Conditional entropy. Mutual information. L65 Dept. of Linguistics, Indiana University Fall 205 Information theory answers two fundamental questions in communication theory: What is the ultimate data compression? What is the transmission rate

More information

Dept. of Linguistics, Indiana University Fall 2015

Dept. of Linguistics, Indiana University Fall 2015 L645 Dept. of Linguistics, Indiana University Fall 2015 1 / 28 Information theory answers two fundamental questions in communication theory: What is the ultimate data compression? What is the transmission

More information

Hiding Data in a QImage File

Hiding Data in a QImage File Hiding Data in a QImage File Gabriela Mogos Abstract The idea of embedding some information within a digital media, in such a way that the inserted data are intrinsically part of the media itself, has

More information

Cryptography and Security Final Exam

Cryptography and Security Final Exam Cryptography and Security Final Exam Solution Serge Vaudenay 29.1.2018 duration: 3h no documents allowed, except one 2-sided sheet of handwritten notes a pocket calculator is allowed communication devices

More information

Chapter 2. Binary and M-ary Hypothesis Testing 2.1 Introduction (Levy 2.1)

Chapter 2. Binary and M-ary Hypothesis Testing 2.1 Introduction (Levy 2.1) Chapter 2. Binary and M-ary Hypothesis Testing 2.1 Introduction (Levy 2.1) Detection problems can usually be casted as binary or M-ary hypothesis testing problems. Applications: This chapter: Simple hypothesis

More information

Essentially Optimal Robust Secret Sharing with Maximal Corruptions

Essentially Optimal Robust Secret Sharing with Maximal Corruptions Essentially Optimal Robust Secret Sharing with Maximal Corruptions Allison Bishop 1, Valerio Pastro 1, Rajmohan Rajaraman 2, and Daniel Wichs 2 1 Columbia University 2 Northeastern University November

More information

Detection theory 101 ELEC-E5410 Signal Processing for Communications

Detection theory 101 ELEC-E5410 Signal Processing for Communications Detection theory 101 ELEC-E5410 Signal Processing for Communications Binary hypothesis testing Null hypothesis H 0 : e.g. noise only Alternative hypothesis H 1 : signal + noise p(x;h 0 ) γ p(x;h 1 ) Trade-off

More information

Digital communication system. Shannon s separation principle

Digital communication system. Shannon s separation principle Digital communication system Representation of the source signal by a stream of (binary) symbols Adaptation to the properties of the transmission channel information source source coder channel coder modulation

More information

Joint Compression and Digital Watermarking: Information-Theoretic Study and Algorithms Development

Joint Compression and Digital Watermarking: Information-Theoretic Study and Algorithms Development Joint Compression and Digital Watermarking: Information-Theoretic Study and Algorithms Development by Wei Sun A thesis presented to the University of Waterloo in fulfillment of the thesis requirement for

More information

PERFECT SECRECY AND ADVERSARIAL INDISTINGUISHABILITY

PERFECT SECRECY AND ADVERSARIAL INDISTINGUISHABILITY PERFECT SECRECY AND ADVERSARIAL INDISTINGUISHABILITY BURTON ROSENBERG UNIVERSITY OF MIAMI Contents 1. Perfect Secrecy 1 1.1. A Perfectly Secret Cipher 2 1.2. Odds Ratio and Bias 3 1.3. Conditions for Perfect

More information

Lecture 8: Shannon s Noise Models

Lecture 8: Shannon s Noise Models Error Correcting Codes: Combinatorics, Algorithms and Applications (Fall 2007) Lecture 8: Shannon s Noise Models September 14, 2007 Lecturer: Atri Rudra Scribe: Sandipan Kundu& Atri Rudra Till now we have

More information

Detection Theory. Chapter 3. Statistical Decision Theory I. Isael Diaz Oct 26th 2010

Detection Theory. Chapter 3. Statistical Decision Theory I. Isael Diaz Oct 26th 2010 Detection Theory Chapter 3. Statistical Decision Theory I. Isael Diaz Oct 26th 2010 Outline Neyman-Pearson Theorem Detector Performance Irrelevant Data Minimum Probability of Error Bayes Risk Multiple

More information

Information Leakage of Correlated Source Coded Sequences over a Channel with an Eavesdropper

Information Leakage of Correlated Source Coded Sequences over a Channel with an Eavesdropper Information Leakage of Correlated Source Coded Sequences over a Channel with an Eavesdropper Reevana Balmahoon and Ling Cheng School of Electrical and Information Engineering University of the Witwatersrand

More information

Energy State Amplification in an Energy Harvesting Communication System

Energy State Amplification in an Energy Harvesting Communication System Energy State Amplification in an Energy Harvesting Communication System Omur Ozel Sennur Ulukus Department of Electrical and Computer Engineering University of Maryland College Park, MD 20742 omur@umd.edu

More information

On Dependence Balance Bounds for Two Way Channels

On Dependence Balance Bounds for Two Way Channels On Dependence Balance Bounds for Two Way Channels Ravi Tandon Sennur Ulukus Department of Electrical and Computer Engineering University of Maryland, College Park, MD 20742 ravit@umd.edu ulukus@umd.edu

More information

Lecture 2. Capacity of the Gaussian channel

Lecture 2. Capacity of the Gaussian channel Spring, 207 5237S, Wireless Communications II 2. Lecture 2 Capacity of the Gaussian channel Review on basic concepts in inf. theory ( Cover&Thomas: Elements of Inf. Theory, Tse&Viswanath: Appendix B) AWGN

More information