Information Hiding Problems: Hiding Capacity and Key Design

Size: px
Start display at page:

Download "Information Hiding Problems: Hiding Capacity and Key Design"

Transcription

1 Iformatio Hidig Problems: Hidig Capacity ad Key Desig Joseph. O Sulliva Electroic Systems ad Sigals Research Laboratory Departmet of Electrical Egieerig Washigto Uiversity i St. Louis Iformatio Hidig Problems Game-Theory Formulatios Eamples Costraits Hidig Capacity Spectrum of problems Private Key Public Coclusios

2 J.. O Sulliva ITW 00 0/4/0 Iformatio Hidig Problems M S Ecoder- Ifo Hider ttac Chael Y Decoder ˆ M K Covertet Key S i K i are pairwise i.i.d.; Source Sets of allowable iformatio hidig ad attac chaels: ad Rate R Ecoder f Decoder Probability of Error: P e R R m P Y K m f S K m

3 J.. O Sulliva ITW 00 0/4/0 Iformatio Hidig Problems M S Ecoder- Ifo Hider ttac Chael Y Decoder ˆ M K Private Game: K = S Public Game: K idepedet of S Rage i betwee: K V S V quatifies iformatio provided about S by K Key selectio: Fi V vary K

4 J.. O Sulliva ITW 00 0/4/0 Iformatio Hidig Games M S Ecoder- Ifo Hider ttac Chael Y Decoder ˆ M K Fi sets of allowable iformatio hidig ad attac chaels: ad Capacity Game: Determie maimal achievable rate R hidig capacity C Mouli ad O Sulliva; Cohe ad Lapidoth; Merhav ad Someh- Baruch; Che Worell ad Barro Error Epoet Game: Determie maimum error epoet for each value of rate R ER Merhav ad Someh-Baruch Other Games: idetificatio detectio ad estimatio figerpritig Mouli et al.; Merhav ad Steiberg; etc

5 J.. O Sulliva ITW 00 0/4/0 Iformatio Hidig Costraits Ecoded data idepedet of covertet Message Ecoder M Source P S Source Ecoder ttac Decoder

6 Stegaography J.. O Sulliva ITW 00 0/4/0 - hidig iformatio so that its presece is udetectable Ecoded data eed ot be idepedet of covertet. Message Ecoder Source P S Source Ecoder Detectio Suppose the attacer simply wats to detect the presece or absece of a hidde message. If D P P S 0 the perfect hidig is achieved. Otherwise the error rate is determied by D P P S.

7 Issues i Embeddig J.. O Sulliva ITW 00 0/4/0 Message Source Ecoder ttac Decoder

8 Message J.. O Sulliva ITW 00 0/4/0 Iformatio Hidig Costraits S Trasparecy or uobtrusiveess: S ad similar mea w.p. ep. boud E[ d S ] D P[ d S D ] 0 S P[ d Source Ecoder ttac Decoder S D S ] e S

9 J.. O Sulliva ITW 00 0/4/0 Eample Implemetatio Issues Trasform Model coefficiets i trasform domai Group coefficiets accordig to margials Iformatio embeddig i coefficiets Perceptually most sigificat Iverse trasform Radomized codig Embeddig Iverse Trasform

10 J.. O Sulliva ITW 00 0/4/0 ttac Chael Costraits ttac Chael Costraits Source Ecoder ttac Decoder Message Robustess: ad Y similar mea w.p. ep. boud S ] [ D Y d E e D Y d P D Y d P ] [ 0 ] [

11 J.. O Sulliva ITW 00 0/4/0 ttac Chael Costraits Message S Source Ecoder ttac - Iadequacy of E[ d Y ] D

12 J.. O Sulliva ITW 00 0/4/0 rbitrary Varyig Chael Result Message S Source Ecoder ttac - If a deletio attac is allowed with probability bouded away from zero the the hidig capacity is zero.

13 J.. O Sulliva ITW 00 0/4/0 ttac Chael Costraits ttac Chael Costraits Mouli ad O Sulliva: mea Memoryless attacs: Bloc-memoryless: Probability oe or large deviatios bouds L i il i L i il L L i i i y y y y e D Y d P D Y d P ] [ 0 ] [ ] [ D Y d E

14 J.. O Sulliva ITW 00 0/4/0 ttac Chael Costraits For may applicatios deletio attacs are used Prisoer or subversive commuicatio For other applicatios deletio is uacceptable Uauthorized use or acquisitio of itellectual property Traitor problems Mathematical models may ot match reality

15 J.. O Sulliva ITW 00 0/4/0 Game Theory View cowledge iformatio will be hidde cowledge eistece of adversary Publish hidig strategy cosistet with stadard approaches to cryptography Need for secret ey for radomized codig Need for ecodig robustess agaist broad families of attacs Need for decoder to adapt to or be robust with respect to attacs

16 J.. O Sulliva ITW 00 0/4/0 Iformatio Hidig Iformatio Hidig Codig Theorems Codig Theorems Public Game: Private Game: Other: Gaussia squared error game: Public ad private games have equal capacity Where ; ; mi ma ; mi ma ; ; mi ma K S U I K Y U I C S Y I C S U I Y U I C

17 J.. O Sulliva ITW 00 0/4/0 Defiitios fiite sets U is a auiliary radom variable over a set of ow cardiality : : D y d q y y s p s u q D s d s p s u s u s u

18 J.. O Sulliva ITW 00 0/4/0 Payoff Fuctio Payoff Fuctio Properties Properties Set is cove for every Set is cove Cove i Cocave i pus Cove i pus Key Desig: Cove i ps ; ; y s u p s u p s u K S U I K Y U I J

19 J.. O Sulliva ITW 00 0/4/0 Payoff Fuctio Relatioship to Chiag ad Cover ISIT 00 Uified view of chael codig with state iformatio ad source codig with side iformatio at the decoder Iformatio available at decoder iformatio available at ecoder I U; Y K I U; S K I U; Y K I U; S K

20 J.. O Sulliva ITW 00 0/4/0 Equal Capacity i Public ad Private Games Barro Che ad Worell 00. If the ** that achieve capacity are such that U Y S is a Marov chai ad the resultig joit distributio o SY is the same as i the public game the the values of the public ad private games are equal. C ma mi I U; Y I U; S

21 J.. O Sulliva ITW 00 0/4/0 Key Desig Issues Key Desig Issues Primary roles of ey: Provide radomizatio of ecodig strategy ecoder-decoder commo radomess Iform decoder about covertet Iformatio: K V S Marov Chai v v V S U I v V Y U I v p v p K S U I K Y U I v s u s u ] ; ; [ ; ;

22 J.. O Sulliva ITW 00 0/4/0 Commets o Keys ribtrary ey K implies select V=fK. Last epasio emphasizes IU;YV-IU;SV Suppose V V S. Let V = f K ad V = f K ad assume the margials o K ad K are idetical. The the capacity of the game with K is at least the capacity of the game with K.

23 J.. O Sulliva ITW 00 0/4/0 Further Implemetatio Issues Distributios o sources Images video voice music Optio: base models o successful compressio algorithms Model real attacs Malicious beig uaticipated Cut-out of itellectual property; edited photos Figerpritig for traitor problems Collusio attacs out of m

24 J.. O Sulliva ITW 00 0/4/0 Related Emergig pplicatios Upgrades of legacy commuicatio systems Worell Ramchadra Pradha et al. Legacy system defies covertet Digital upgrade is ecoded data Importace of Gaussia view DC-IM etc.

25 J.. O Sulliva ITW 00 0/4/0 Coclusios Reviewed capacity results i iformatio hidig games Reviewed some roles of eys For more iformatio o Our results see Mouli ad O Sulliva Error epoet games see Merhav ad Someh-Baruch Gaussia games see Cohe ad Lapidoth lso: Worell Ramchadra Steiberg several special issues cofereces etc.

The Center for Security Technologies Academic Issues in Homeland Security

The Center for Security Technologies Academic Issues in Homeland Security The Ceter for Security Techologies Academic Issues i Homelad Security Joseph A. O Sulliva Professor of Electrical Egieerig Director of the Electroic Systems ad Sigals Research Laboratory Associate Director

More information

Asymptotic Coupling and Its Applications in Information Theory

Asymptotic Coupling and Its Applications in Information Theory Asymptotic Couplig ad Its Applicatios i Iformatio Theory Vicet Y. F. Ta Joit Work with Lei Yu Departmet of Electrical ad Computer Egieerig, Departmet of Mathematics, Natioal Uiversity of Sigapore IMS-APRM

More information

Large Deviations Performance Analysis for Biometrics Recognition

Large Deviations Performance Analysis for Biometrics Recognition J.A. O Sulliva, 2002 Allerto Coferece Large Deviatios erformace Aalysis for Biometrics Recogitio Joseph A. O Sulliva ad Natalia Schmid Electroic Systems ad Sigals Research Laboratory Departmet of Electrical

More information

Lecture 6: Source coding, Typicality, and Noisy channels and capacity

Lecture 6: Source coding, Typicality, and Noisy channels and capacity 15-859: Iformatio Theory ad Applicatios i TCS CMU: Sprig 2013 Lecture 6: Source codig, Typicality, ad Noisy chaels ad capacity Jauary 31, 2013 Lecturer: Mahdi Cheraghchi Scribe: Togbo Huag 1 Recap Uiversal

More information

Run-length & Entropy Coding. Redundancy Removal. Sampling. Quantization. Perform inverse operations at the receiver EEE

Run-length & Entropy Coding. Redundancy Removal. Sampling. Quantization. Perform inverse operations at the receiver EEE Geeral e Image Coder Structure Motio Video (s 1,s 2,t) or (s 1,s 2 ) Natural Image Samplig A form of data compressio; usually lossless, but ca be lossy Redudacy Removal Lossless compressio: predictive

More information

Covert Communication with Noncausal Channel-State Information at the Transmitter

Covert Communication with Noncausal Channel-State Information at the Transmitter Covert Commuicatio with Nocausal Chael-State Iformatio at the Trasmitter Si-Hyeo Lee, Ligog Wag, Ashish Khisti, ad Gregory W. Worell POSTECH, Pohag, South Korea (e-mail:sihyeo@postech.ac.kr ETIS Uiversité

More information

Entropies & Information Theory

Entropies & Information Theory Etropies & Iformatio Theory LECTURE I Nilajaa Datta Uiversity of Cambridge,U.K. For more details: see lecture otes (Lecture 1- Lecture 5) o http://www.qi.damtp.cam.ac.uk/ode/223 Quatum Iformatio Theory

More information

Information Theory and Coding

Information Theory and Coding Sol. Iformatio Theory ad Codig. The capacity of a bad-limited additive white Gaussia (AWGN) chael is give by C = Wlog 2 ( + σ 2 W ) bits per secod(bps), where W is the chael badwidth, is the average power

More information

Channel coding, linear block codes, Hamming and cyclic codes Lecture - 8

Channel coding, linear block codes, Hamming and cyclic codes Lecture - 8 Digital Commuicatio Chael codig, liear block codes, Hammig ad cyclic codes Lecture - 8 Ir. Muhamad Asial, MSc., PhD Ceter for Iformatio ad Commuicatio Egieerig Research (CICER) Electrical Egieerig Departmet

More information

A Partial Decode-Forward Scheme For A Network with N relays

A Partial Decode-Forward Scheme For A Network with N relays A Partial Decode-Forward Scheme For A etwork with relays Yao Tag ECE Departmet, McGill Uiversity Motreal, QC, Caada Email: yaotag2@mailmcgillca Mai Vu ECE Departmet, Tufts Uiversity Medford, MA, USA Email:

More information

Resilient Source Coding

Resilient Source Coding Resiliet Source Codig Maël Le Treust Laboratoire des Sigaux et Systèmes CNRS - Supélec - Uiv. Paris Sud 11 91191, Gif-sur-Yvette, Frace Email: mael.letreust@lss.supelec.fr Samso Lasaulce Laboratoire des

More information

Lecture 7: Channel coding theorem for discrete-time continuous memoryless channel

Lecture 7: Channel coding theorem for discrete-time continuous memoryless channel Lecture 7: Chael codig theorem for discrete-time cotiuous memoryless chael Lectured by Dr. Saif K. Mohammed Scribed by Mirsad Čirkić Iformatio Theory for Wireless Commuicatio ITWC Sprig 202 Let us first

More information

Sensitivity Analysis of Daubechies 4 Wavelet Coefficients for Reduction of Reconstructed Image Error

Sensitivity Analysis of Daubechies 4 Wavelet Coefficients for Reduction of Reconstructed Image Error Proceedigs of the 6th WSEAS Iteratioal Coferece o SIGNAL PROCESSING, Dallas, Texas, USA, March -4, 7 67 Sesitivity Aalysis of Daubechies 4 Wavelet Coefficiets for Reductio of Recostructed Image Error DEVINDER

More information

EECS564 Estimation, Filtering, and Detection Hwk 2 Solns. Winter p θ (z) = (2θz + 1 θ), 0 z 1

EECS564 Estimation, Filtering, and Detection Hwk 2 Solns. Winter p θ (z) = (2θz + 1 θ), 0 z 1 EECS564 Estimatio, Filterig, ad Detectio Hwk 2 Sols. Witer 25 4. Let Z be a sigle observatio havig desity fuctio where. p (z) = (2z + ), z (a) Assumig that is a oradom parameter, fid ad plot the maximum

More information

OFDM Precoder for Minimizing BER Upper Bound of MLD under Imperfect CSI

OFDM Precoder for Minimizing BER Upper Bound of MLD under Imperfect CSI MIMO-OFDM OFDM Precoder for Miimizig BER Upper Boud of MLD uder Imperfect CSI MCRG Joit Semiar Jue the th 008 Previously preseted at ICC 008 Beijig o May the st 008 Boosar Pitakdumrogkija Kazuhiko Fukawa

More information

Linear Algebra Issues in Wireless Communications

Linear Algebra Issues in Wireless Communications Rome-Moscow school of Matrix Methods ad Applied Liear Algebra August 0 September 18, 016 Liear Algebra Issues i Wireless Commuicatios Russia Research Ceter [vladimir.lyashev@huawei.com] About me ead of

More information

Universal source coding for complementary delivery

Universal source coding for complementary delivery SITA2006 i Hakodate 2005.2. p. Uiversal source codig for complemetary delivery Akisato Kimura, 2, Tomohiko Uyematsu 2, Shigeaki Kuzuoka 2 Media Iformatio Laboratory, NTT Commuicatio Sciece Laboratories,

More information

Lecture 14: Graph Entropy

Lecture 14: Graph Entropy 15-859: Iformatio Theory ad Applicatios i TCS Sprig 2013 Lecture 14: Graph Etropy March 19, 2013 Lecturer: Mahdi Cheraghchi Scribe: Euiwoog Lee 1 Recap Bergma s boud o the permaet Shearer s Lemma Number

More information

OPTIMAL PIECEWISE UNIFORM VECTOR QUANTIZATION OF THE MEMORYLESS LAPLACIAN SOURCE

OPTIMAL PIECEWISE UNIFORM VECTOR QUANTIZATION OF THE MEMORYLESS LAPLACIAN SOURCE Joural of ELECTRICAL EGIEERIG, VOL. 56, O. 7-8, 2005, 200 204 OPTIMAL PIECEWISE UIFORM VECTOR QUATIZATIO OF THE MEMORYLESS LAPLACIA SOURCE Zora H. Perić Veljo Lj. Staović Alesadra Z. Jovaović Srdja M.

More information

Fundamentals. Relative data redundancy of the set. C.E., NCU, Taiwan Angela Chih-Wei Tang,

Fundamentals. Relative data redundancy of the set. C.E., NCU, Taiwan Angela Chih-Wei Tang, Image Compressio Agela Chih-Wei Tag ( 唐之瑋 ) Departmet of Commuicatio Egieerig Natioal Cetral Uiversity JhogLi, Taiwa 2012 Sprig Fudametals Compressio ratio C R / 2, 1 : origial, 2 1 : compressed Relative

More information

On Successive Refinement for the Wyner-Ziv Problem with Partially Cooperating Decoders

On Successive Refinement for the Wyner-Ziv Problem with Partially Cooperating Decoders ISIT 2008, Toroto, Caada, July 6-11, 2008 O Successive Refiemet for the Wyer-Ziv Problem with Partially Cooperatig Decoders Shraga I. Bross ad Tsachy Weissma, School of Egieerig, Bar-Ila Uiversity, Ramat-Ga

More information

Integer Programming (IP)

Integer Programming (IP) Iteger Programmig (IP) The geeral liear mathematical programmig problem where Mied IP Problem - MIP ma c T + h Z T y A + G y + y b R p + vector of positive iteger variables y vector of positive real variables

More information

Pattern Classification

Pattern Classification Patter Classificatio All materials i these slides were tae from Patter Classificatio (d ed) by R. O. Duda, P. E. Hart ad D. G. Stor, Joh Wiley & Sos, 000 with the permissio of the authors ad the publisher

More information

Entropy and Ergodic Theory Lecture 5: Joint typicality and conditional AEP

Entropy and Ergodic Theory Lecture 5: Joint typicality and conditional AEP Etropy ad Ergodic Theory Lecture 5: Joit typicality ad coditioal AEP 1 Notatio: from RVs back to distributios Let (Ω, F, P) be a probability space, ad let X ad Y be A- ad B-valued discrete RVs, respectively.

More information

Refinement of Two Fundamental Tools in Information Theory

Refinement of Two Fundamental Tools in Information Theory Refiemet of Two Fudametal Tools i Iformatio Theory Raymod W. Yeug Istitute of Network Codig The Chiese Uiversity of Hog Kog Joit work with Siu Wai Ho ad Sergio Verdu Discotiuity of Shao s Iformatio Measures

More information

First Year Quantitative Comp Exam Spring, Part I - 203A. f X (x) = 0 otherwise

First Year Quantitative Comp Exam Spring, Part I - 203A. f X (x) = 0 otherwise First Year Quatitative Comp Exam Sprig, 2012 Istructio: There are three parts. Aswer every questio i every part. Questio I-1 Part I - 203A A radom variable X is distributed with the margial desity: >

More information

EE260: Digital Design, Spring n MUX Gate n Rudimentary functions n Binary Decoders. n Binary Encoders n Priority Encoders

EE260: Digital Design, Spring n MUX Gate n Rudimentary functions n Binary Decoders. n Binary Encoders n Priority Encoders EE260: Digital Desig, Sprig 2018 EE 260: Itroductio to Digital Desig MUXs, Ecoders, Decoders Yao Zheg Departmet of Electrical Egieerig Uiversity of Hawaiʻi at Māoa Overview of Ecoder ad Decoder MUX Gate

More information

CS434a/541a: Pattern Recognition Prof. Olga Veksler. Lecture 5

CS434a/541a: Pattern Recognition Prof. Olga Veksler. Lecture 5 CS434a/54a: Patter Recogitio Prof. Olga Veksler Lecture 5 Today Itroductio to parameter estimatio Two methods for parameter estimatio Maimum Likelihood Estimatio Bayesia Estimatio Itroducto Bayesia Decisio

More information

1 Inferential Methods for Correlation and Regression Analysis

1 Inferential Methods for Correlation and Regression Analysis 1 Iferetial Methods for Correlatio ad Regressio Aalysis I the chapter o Correlatio ad Regressio Aalysis tools for describig bivariate cotiuous data were itroduced. The sample Pearso Correlatio Coefficiet

More information

The source coding game with a cheating switcher

The source coding game with a cheating switcher The source codig game with a cheatig switcher Hari Palaiyaur hpalaiya@eecsberkeleyedu Cheg Chag cchag@eecsberkeleyedu Departmet of Electrical Egieerig ad Computer Scieces Uiversity of Califoria at Berkeley

More information

Capacity-distortion trade-off in channels with state

Capacity-distortion trade-off in channels with state Forty-Eighth Aual Allerto Coferece Allerto House, UIUC, Illiois, USA September 29 - October, 200 Capacity-distortio trade-off i chaels with state Chirajib Choudhuri, Youg-Ha Kim ad Urbashi Mitra Abstract

More information

Capacity of Compound State-Dependent Channels with States Known at the Transmitter

Capacity of Compound State-Dependent Channels with States Known at the Transmitter Capacity of Compoud State-Depedet Chaels with States Kow at the Trasmitter Pablo Piataida Departmet of Telecommuicatios SUPELEC Plateau de Moulo, 992 Gif-sur-Yvette, Frace Email: pablo.piataida@supelec.fr

More information

Fig. 2. Block Diagram of a DCS

Fig. 2. Block Diagram of a DCS Iformatio source Optioal Essetial From other sources Spread code ge. Format A/D Source ecode Ecrypt Auth. Chael ecode Pulse modu. Multiplex Badpass modu. Spread spectrum modu. X M m i Digital iput Digital

More information

SUCCESSIVE INTERFERENCE CANCELLATION DECODING FOR THE K -USER CYCLIC INTERFERENCE CHANNEL

SUCCESSIVE INTERFERENCE CANCELLATION DECODING FOR THE K -USER CYCLIC INTERFERENCE CHANNEL Joural of Theoretical ad Applied Iformatio Techology 31 st December 212 Vol 46 No2 25-212 JATIT & LLS All rights reserved ISSN: 1992-8645 wwwatitorg E-ISSN: 1817-3195 SCCESSIVE INTERFERENCE CANCELLATION

More information

4. Basic probability theory

4. Basic probability theory Cotets Basic cocepts Discrete radom variables Discrete distributios (br distributios) Cotiuous radom variables Cotiuous distributios (time distributios) Other radom variables Lect04.ppt S-38.45 - Itroductio

More information

Second-Order Rate of Constant-Composition Codes for the Gel fand-pinsker Channel

Second-Order Rate of Constant-Composition Codes for the Gel fand-pinsker Channel Research Collectio Coferece aper Secod-Order Rate of Costat-Compositio Codes for the Gel fad-isker Chael Author(s: Scarlett, Joatha ublicatio Date: 204 ermaet ik: https://doi.org/0.3929/ethz-a-0009456

More information

Cooperative Communication Fundamentals & Coding Techniques

Cooperative Communication Fundamentals & Coding Techniques 3 th ICACT Tutorial Cooperative commuicatio fudametals & codig techiques Cooperative Commuicatio Fudametals & Codig Techiques 0..4 Electroics ad Telecommuicatio Research Istitute Kiug Jug 3 th ICACT Tutorial

More information

Multiterminal source coding with complementary delivery

Multiterminal source coding with complementary delivery Iteratioal Symposium o Iformatio Theory ad its Applicatios, ISITA2006 Seoul, Korea, October 29 November 1, 2006 Multitermial source codig with complemetary delivery Akisato Kimura ad Tomohiko Uyematsu

More information

Lecture 01: the Central Limit Theorem. 1 Central Limit Theorem for i.i.d. random variables

Lecture 01: the Central Limit Theorem. 1 Central Limit Theorem for i.i.d. random variables CSCI-B609: A Theorist s Toolkit, Fall 06 Aug 3 Lecture 0: the Cetral Limit Theorem Lecturer: Yua Zhou Scribe: Yua Xie & Yua Zhou Cetral Limit Theorem for iid radom variables Let us say that we wat to aalyze

More information

Report on Private Information Retrieval over Unsynchronized Databases

Report on Private Information Retrieval over Unsynchronized Databases Report o Private Iformatio Retrieval over Usychroized Databases Lembit Valgma Supervised by Vitaly Skachek May 25, 217 1 Problem Statemet There are may challeges cocerig olie privacy. Private iformatio

More information

CEE 522 Autumn Uncertainty Concepts for Geotechnical Engineering

CEE 522 Autumn Uncertainty Concepts for Geotechnical Engineering CEE 5 Autum 005 Ucertaity Cocepts for Geotechical Egieerig Basic Termiology Set A set is a collectio of (mutually exclusive) objects or evets. The sample space is the (collectively exhaustive) collectio

More information

Symmetric Two-User Gaussian Interference Channel with Common Messages

Symmetric Two-User Gaussian Interference Channel with Common Messages Symmetric Two-User Gaussia Iterferece Chael with Commo Messages Qua Geg CSL ad Dept. of ECE UIUC, IL 680 Email: geg5@illiois.edu Tie Liu Dept. of Electrical ad Computer Egieerig Texas A&M Uiversity, TX

More information

Non-Asymptotic Achievable Rates for Gaussian Energy-Harvesting Channels: Best-Effort and Save-and-Transmit

Non-Asymptotic Achievable Rates for Gaussian Energy-Harvesting Channels: Best-Effort and Save-and-Transmit 08 IEEE Iteratioal Symposium o Iformatio Theory ISIT No-Asymptotic Achievable Rates for Gaussia Eergy-Harvestig Chaels: Best-Effort ad Save-ad-Trasmit Silas L Fog Departmet of Electrical ad Computer Egieerig

More information

Lecture 15: Strong, Conditional, & Joint Typicality

Lecture 15: Strong, Conditional, & Joint Typicality EE376A/STATS376A Iformatio Theory Lecture 15-02/27/2018 Lecture 15: Strog, Coditioal, & Joit Typicality Lecturer: Tsachy Weissma Scribe: Nimit Sohoi, William McCloskey, Halwest Mohammad I this lecture,

More information

The random version of Dvoretzky s theorem in l n

The random version of Dvoretzky s theorem in l n The radom versio of Dvoretzky s theorem i l Gideo Schechtma Abstract We show that with high probability a sectio of the l ball of dimesio k cε log c > 0 a uiversal costat) is ε close to a multiple of the

More information

Quick Review of Probability

Quick Review of Probability Quick Review of Probability Berli Che Departmet of Computer Sciece & Iformatio Egieerig Natioal Taiwa Normal Uiversity Refereces: 1. W. Navidi. Statistics for Egieerig ad Scietists. Chapter & Teachig Material.

More information

Quick Review of Probability

Quick Review of Probability Quick Review of Probability Berli Che Departmet of Computer Sciece & Iformatio Egieerig Natioal Taiwa Normal Uiversity Refereces: 1. W. Navidi. Statistics for Egieerig ad Scietists. Chapter 2 & Teachig

More information

A statistical method to determine sample size to estimate characteristic value of soil parameters

A statistical method to determine sample size to estimate characteristic value of soil parameters A statistical method to determie sample size to estimate characteristic value of soil parameters Y. Hojo, B. Setiawa 2 ad M. Suzuki 3 Abstract Sample size is a importat factor to be cosidered i determiig

More information

Overview of Gaussian MIMO (Vector) BC

Overview of Gaussian MIMO (Vector) BC Overview of Gaussia MIMO (Vector) BC Gwamo Ku Adaptive Sigal Processig ad Iformatio Theory Research Group Nov. 30, 2012 Outlie / Capacity Regio of Gaussia MIMO BC System Structure Kow Capacity Regios -

More information

arxiv: v1 [cs.it] 13 Jul 2012

arxiv: v1 [cs.it] 13 Jul 2012 O the Sum Capacity of the Discrete Memoryless Iterferece Chael with Oe-Sided Weak Iterferece ad Mixed Iterferece Fagfag Zhu ad Biao Che Syracuse Uiversity Departmet of EECS Syracuse, NY 3244 Email: fazhu{biche}@syr.edu

More information

A More General Version of the Costa Theorem

A More General Version of the Costa Theorem Joural of ommuicatio Egieerig Vol No 4 utum 03 33 More Geeral Versio of the osta Theorem Nima zabi-nezhad Ghosheh bed Hodtai ad Mohammad Molavi Kakhki Departmet of Electrical Egieerig Ferdowsi Uiversity

More information

A meta-converse for private communication over quantum channels

A meta-converse for private communication over quantum channels A meta-coverse for private commuicatio over quatum chaels Mario Berta with Mark M. Wilde ad Marco Tomamichel IEEE Trasactios o Iformatio Theory, 63(3), 1792 1817 (2017) Beyod IID Sigapore - July 17, 2017

More information

Four-dimensional Vector Matrix Determinant and Inverse

Four-dimensional Vector Matrix Determinant and Inverse I.J. Egieerig ad Maufacturig 013 30-37 Published Olie Jue 01 i MECS (http://www.mecs-press.et) DOI: 10.5815/iem.01.03.05 vailable olie at http://www.mecs-press.et/iem Four-dimesioal Vector Matrix Determiat

More information

Finite Block-Length Gains in Distributed Source Coding

Finite Block-Length Gains in Distributed Source Coding Decoder Fiite Block-Legth Gais i Distributed Source Codig Farhad Shirai EECS Departmet Uiversity of Michiga A Arbor,USA Email: fshirai@umichedu S Sadeep Pradha EECS Departmet Uiversity of Michiga A Arbor,USA

More information

This is an introductory course in Analysis of Variance and Design of Experiments.

This is an introductory course in Analysis of Variance and Design of Experiments. 1 Notes for M 384E, Wedesday, Jauary 21, 2009 (Please ote: I will ot pass out hard-copy class otes i future classes. If there are writte class otes, they will be posted o the web by the ight before class

More information

ECE 901 Lecture 14: Maximum Likelihood Estimation and Complexity Regularization

ECE 901 Lecture 14: Maximum Likelihood Estimation and Complexity Regularization ECE 90 Lecture 4: Maximum Likelihood Estimatio ad Complexity Regularizatio R Nowak 5/7/009 Review : Maximum Likelihood Estimatio We have iid observatios draw from a ukow distributio Y i iid p θ, i,, where

More information

An Introduction to Asymptotic Theory

An Introduction to Asymptotic Theory A Itroductio to Asymptotic Theory Pig Yu School of Ecoomics ad Fiace The Uiversity of Hog Kog Pig Yu (HKU) Asymptotic Theory 1 / 20 Five Weapos i Asymptotic Theory Five Weapos i Asymptotic Theory Pig Yu

More information

Journal of Multivariate Analysis. Superefficient estimation of the marginals by exploiting knowledge on the copula

Journal of Multivariate Analysis. Superefficient estimation of the marginals by exploiting knowledge on the copula Joural of Multivariate Aalysis 102 (2011) 1315 1319 Cotets lists available at ScieceDirect Joural of Multivariate Aalysis joural homepage: www.elsevier.com/locate/jmva Superefficiet estimatio of the margials

More information

Oblivious Transfer using Elliptic Curves

Oblivious Transfer using Elliptic Curves Oblivious Trasfer usig Elliptic Curves bhishek Parakh Louisiaa State Uiversity, ato Rouge, L May 4, 006 bstract: This paper proposes a algorithm for oblivious trasfer usig elliptic curves lso, we preset

More information

Vector Permutation Code Design Algorithm. Danilo SILVA and Weiler A. FINAMORE

Vector Permutation Code Design Algorithm. Danilo SILVA and Weiler A. FINAMORE Iteratioal Symposium o Iformatio Theory ad its Applicatios, ISITA2004 Parma, Italy, October 10 13, 2004 Vector Permutatio Code Desig Algorithm Dailo SILVA ad Weiler A. FINAMORE Cetro de Estudos em Telecomuicações

More information

On Evaluating the Rate-Distortion Function of Sources with Feed-Forward and the Capacity of Channels with Feedback.

On Evaluating the Rate-Distortion Function of Sources with Feed-Forward and the Capacity of Channels with Feedback. O Evaluatig the Rate-Distortio Fuctio of Sources with Feed-Forward ad the Capacity of Chaels with Feedback. Ramji Vekataramaa ad S. Sadeep Pradha Departmet of EECS, Uiversity of Michiga, A Arbor, MI 4805

More information

Finite Difference Approximation for Transport Equation with Shifts Arising in Neuronal Variability

Finite Difference Approximation for Transport Equation with Shifts Arising in Neuronal Variability Iteratioal Joural of Sciece ad Research (IJSR) ISSN (Olie): 39-764 Ide Copericus Value (3): 64 Impact Factor (3): 4438 Fiite Differece Approimatio for Trasport Equatio with Shifts Arisig i Neuroal Variability

More information

Volume 3, Number 2, 2017 Pages Jordan Journal of Electrical Engineering ISSN (Print): , ISSN (Online):

Volume 3, Number 2, 2017 Pages Jordan Journal of Electrical Engineering ISSN (Print): , ISSN (Online): JJEE Volume 3, Number, 07 Pages 50-58 Jorda Joural of Electrical Egieerig ISSN (Prit: 409-9600, ISSN (Olie: 409-969 Liftig Based S-Box for Scalable Bloc Cipher Desig Based o Filter Bas Saleh S. Saraireh

More information

Random Variables, Sampling and Estimation

Random Variables, Sampling and Estimation Chapter 1 Radom Variables, Samplig ad Estimatio 1.1 Itroductio This chapter will cover the most importat basic statistical theory you eed i order to uderstad the ecoometric material that will be comig

More information

ECE 901 Lecture 12: Complexity Regularization and the Squared Loss

ECE 901 Lecture 12: Complexity Regularization and the Squared Loss ECE 90 Lecture : Complexity Regularizatio ad the Squared Loss R. Nowak 5/7/009 I the previous lectures we made use of the Cheroff/Hoeffdig bouds for our aalysis of classifier errors. Hoeffdig s iequality

More information

Unbiased Estimation. February 7-12, 2008

Unbiased Estimation. February 7-12, 2008 Ubiased Estimatio February 7-2, 2008 We begi with a sample X = (X,..., X ) of radom variables chose accordig to oe of a family of probabilities P θ where θ is elemet from the parameter space Θ. For radom

More information

The Gaussian Channel with Noisy Feedback

The Gaussian Channel with Noisy Feedback The Gaussia Chael with Noisy Feedback Youg-Ha Kim Departmet of ECE Uiversity of Califoria, Sa Diego La Jolla, CA 9093, USA yhk@ucsd.edu Amos Lapidoth Sigal ad Iformatio Processig Lab. ETH Zurich 809 Zurich,

More information

Lecture 10: Universal coding and prediction

Lecture 10: Universal coding and prediction 0-704: Iformatio Processig ad Learig Sprig 0 Lecture 0: Uiversal codig ad predictio Lecturer: Aarti Sigh Scribes: Georg M. Goerg Disclaimer: These otes have ot bee subjected to the usual scrutiy reserved

More information

Finite Difference Approximation for First- Order Hyperbolic Partial Differential Equation Arising in Neuronal Variability with Shifts

Finite Difference Approximation for First- Order Hyperbolic Partial Differential Equation Arising in Neuronal Variability with Shifts Iteratioal Joural of Scietific Egieerig ad Research (IJSER) wwwiseri ISSN (Olie): 347-3878, Impact Factor (4): 35 Fiite Differece Approimatio for First- Order Hyperbolic Partial Differetial Equatio Arisig

More information

Compound conditional source coding, Slepian-Wolf list decoding, and applications to media coding

Compound conditional source coding, Slepian-Wolf list decoding, and applications to media coding Compoud coditioal source codig, Slepia-Wolf list decodig, ad applicatios to media codig Stark C. Draper Mitsubishi Electric Research Laboratories Cambridge, MA 02139 USA draper@merl.com PSfrag replacemets

More information

Differential Entropy

Differential Entropy School o Iormatio Sciece Dieretial Etropy 009 - Course - Iormatio Theory - Tetsuo Asao ad Tad matsumoto Email: {t-asao matumoto}@jaist.ac.jp Japa Advaced Istitute o Sciece ad Techology Asahidai - Nomi

More information

Agnostic Learning and Concentration Inequalities

Agnostic Learning and Concentration Inequalities ECE901 Sprig 2004 Statistical Regularizatio ad Learig Theory Lecture: 7 Agostic Learig ad Cocetratio Iequalities Lecturer: Rob Nowak Scribe: Aravid Kailas 1 Itroductio 1.1 Motivatio I the last lecture

More information

Increasing timing capacity using packet coloring

Increasing timing capacity using packet coloring 003 Coferece o Iformatio Scieces ad Systems, The Johs Hopkis Uiversity, March 4, 003 Icreasig timig capacity usig packet colorig Xi Liu ad R Srikat[] Coordiated Sciece Laboratory Uiversity of Illiois e-mail:

More information

Inequalities for Entropies of Sets of Subsets of Random Variables

Inequalities for Entropies of Sets of Subsets of Random Variables Iequalities for Etropies of Sets of Subsets of Radom Variables Chao Tia AT&T Labs-Research Florham Par, NJ 0792, USA. tia@research.att.com Abstract Ha s iequality o the etropy rates of subsets of radom

More information

The estimation of the m parameter of the Nakagami distribution

The estimation of the m parameter of the Nakagami distribution The estimatio of the m parameter of the Nakagami distributio Li-Fei Huag Mig Chua Uiversity Dept. of Applied Stat. ad Ifo. Sciece 5 Teh-Mig Rd., Gwei-Sha Taoyua City Taiwa lhuag@mail.mcu.edu.tw Je-Je Li

More information

SDS 321: Introduction to Probability and Statistics

SDS 321: Introduction to Probability and Statistics SDS 321: Itroductio to Probability ad Statistics Lecture 23: Cotiuous radom variables- Iequalities, CLT Puramrita Sarkar Departmet of Statistics ad Data Sciece The Uiversity of Texas at Austi www.cs.cmu.edu/

More information

Pattern Classification

Pattern Classification Patter Classificatio All materials i these slides were tae from Patter Classificatio (d ed) by R. O. Duda, P. E. Hart ad D. G. Stor, Joh Wiley & Sos, 000 with the permissio of the authors ad the publisher

More information

The Likelihood Encoder with Applications to Lossy Compression and Secrecy

The Likelihood Encoder with Applications to Lossy Compression and Secrecy The Likelihood Ecoder with Applicatios to Lossy Compressio ad Secrecy Eva C. Sog Paul Cuff H. Vicet Poor Dept. of Electrical Eg., Priceto Uiversity, NJ 8544 {csog, cuff, poor}@priceto.edu Abstract A likelihood

More information

Inseparability of the Multiple Access Wiretap Channel

Inseparability of the Multiple Access Wiretap Channel Iseparability of the Multiple Access Wiretap Chael Jiawei Xie Seur Ulukus Departmet of Electrical ad Computer Egieerig Uiversity of Marylad, College Park, MD 074 xiejw@umd.edu ulukus@umd.edu Abstract We

More information

Digital Video Systems ECE 634

Digital Video Systems ECE 634 Digital Video Systems ECE 634 egieerig.purdue.edu/~reibma/ece634/idex.html Professor Amy Reibma MSEE 356 reibma@purdue.edu MulGple descripgo codig Layered codig vs. MDC Layered codig: Base layer is high

More information

A New Achievability Scheme for the Relay Channel

A New Achievability Scheme for the Relay Channel A New Achievability Scheme for the Relay Chael Wei Kag Seur Ulukus Departmet of Electrical ad Computer Egieerig Uiversity of Marylad, College Park, MD 20742 wkag@umd.edu ulukus@umd.edu October 4, 2007

More information

Lecture 27. Capacity of additive Gaussian noise channel and the sphere packing bound

Lecture 27. Capacity of additive Gaussian noise channel and the sphere packing bound Lecture 7 Ageda for the lecture Gaussia chael with average power costraits Capacity of additive Gaussia oise chael ad the sphere packig boud 7. Additive Gaussia oise chael Up to this poit, we have bee

More information

MASSACHUSETTS INSTITUTE OF TECHNOLOGY Department of Electrical Engineering and Computer Science. BACKGROUND EXAM September 30, 2004.

MASSACHUSETTS INSTITUTE OF TECHNOLOGY Department of Electrical Engineering and Computer Science. BACKGROUND EXAM September 30, 2004. MASSACHUSETTS INSTITUTE OF TECHNOLOGY Departmet of Electrical Egieerig ad Computer Sciece 6.34 Discrete Time Sigal Processig Fall 24 BACKGROUND EXAM September 3, 24. Full Name: Note: This exam is closed

More information

The Binary Energy Harvesting Channel with On-Off Fading

The Binary Energy Harvesting Channel with On-Off Fading The Biary Eergy Harvestig Chael with O-Off Fadig Omur Ozel, Kaya Tutucuoglu 2, Seur Ulukus, ad Ayli Yeer 2 Deartmet of Electrical ad Comuter Egieerig, Uiversity of Marylad, College Park, MD 20742 2 Deartmet

More information

Statistical Fundamentals and Control Charts

Statistical Fundamentals and Control Charts Statistical Fudametals ad Cotrol Charts 1. Statistical Process Cotrol Basics Chace causes of variatio uavoidable causes of variatios Assigable causes of variatio large variatios related to machies, materials,

More information

Lecture 27: Optimal Estimators and Functional Delta Method

Lecture 27: Optimal Estimators and Functional Delta Method Stat210B: Theoretical Statistics Lecture Date: April 19, 2007 Lecture 27: Optimal Estimators ad Fuctioal Delta Method Lecturer: Michael I. Jorda Scribe: Guilherme V. Rocha 1 Achievig Optimal Estimators

More information

ε-capacity and Strong Converse for Channels with General State

ε-capacity and Strong Converse for Channels with General State ε-capacity ad Strog Coverse for Chaels with Geeral State Marco Tomamichel Cetre for Quatum Techologies, Natioal Uiversity of Sigapore Email: cqtmarco@us.edu.sg Vicet Y. F. Ta Istitute for Ifocomm Research,

More information

Source Coding With Side Information

Source Coding With Side Information Source Codig With Side Iformatio O. Simeoe NJIT Natioal Chiao-Tug Uiversity, Ja. 2015 O. Simeoe Source Codig With Side Iformatio 1 / 241 Overview Classical problem (1973-) with may applicatios This lecture:

More information

Lossy Compression via Sparse Linear Regression: Computationally Efficient Encoding and Decoding

Lossy Compression via Sparse Linear Regression: Computationally Efficient Encoding and Decoding ossy Compressio via Sparse iear Regressio: Computatioally Efficiet Ecodig ad Decodig Ramji Vekataramaa Dept of Egieerig, Uiv of Cambridge ramjiv@egcamacuk Tuhi Sarkar Dept of EE, IIT Bombay tuhi91@gmailcom

More information

Non-Asymptotic Achievable Rates for Gaussian Energy-Harvesting Channels: Best-Effort and Save-and-Transmit

Non-Asymptotic Achievable Rates for Gaussian Energy-Harvesting Channels: Best-Effort and Save-and-Transmit No-Asymptotic Achievable Rates for Gaussia Eergy-Harvestig Chaels: Best-Effort ad Save-ad-Trasmit Silas L. Fog, Jig Yag, ad Ayli Yeer arxiv:805.089v [cs.it] 30 May 08 Abstract A additive white Gaussia

More information

RAINFALL PREDICTION BY WAVELET DECOMPOSITION

RAINFALL PREDICTION BY WAVELET DECOMPOSITION RAIFALL PREDICTIO BY WAVELET DECOMPOSITIO A. W. JAYAWARDEA Departmet of Civil Egieerig, The Uiversit of Hog Kog, Hog Kog, Chia P. C. XU Academ of Mathematics ad Sstem Scieces, Chiese Academ of Scieces,

More information

Chapter 6 Sampling Distributions

Chapter 6 Sampling Distributions Chapter 6 Samplig Distributios 1 I most experimets, we have more tha oe measuremet for ay give variable, each measuremet beig associated with oe radomly selected a member of a populatio. Hece we eed to

More information

MBACATÓLICA. Quantitative Methods. Faculdade de Ciências Económicas e Empresariais UNIVERSIDADE CATÓLICA PORTUGUESA 9. SAMPLING DISTRIBUTIONS

MBACATÓLICA. Quantitative Methods. Faculdade de Ciências Económicas e Empresariais UNIVERSIDADE CATÓLICA PORTUGUESA 9. SAMPLING DISTRIBUTIONS MBACATÓLICA Quatitative Methods Miguel Gouveia Mauel Leite Moteiro Faculdade de Ciêcias Ecoómicas e Empresariais UNIVERSIDADE CATÓLICA PORTUGUESA 9. SAMPLING DISTRIBUTIONS MBACatólica 006/07 Métodos Quatitativos

More information

2D DSP Basics: 2D Systems

2D DSP Basics: 2D Systems - Digital Image Processig ad Compressio D DSP Basics: D Systems D Systems T[ ] y = T [ ] Liearity Additivity: If T y = T [ ] The + T y = y + y Homogeeity: If The T y = T [ ] a T y = ay = at [ ] Liearity

More information

Sample questions. 8. Let X denote a continuous random variable with probability density function f(x) = 4x 3 /15 for

Sample questions. 8. Let X denote a continuous random variable with probability density function f(x) = 4x 3 /15 for Sample questios Suppose that humas ca have oe of three bloodtypes: A, B, O Assume that 40% of the populatio has Type A, 50% has type B, ad 0% has Type O If a perso has type A, the probability that they

More information

Universal Estimation of Directed Information

Universal Estimation of Directed Information Uiversal Estimatio of Directed Iformatio Lei Zhao, Haim ermuter, Youg-Ha Kim, ad Tsachy Weissma Abstract I this paper, we develop a uiversal algorithm to estimate Massey s directed iformatio for statioary

More information

Topic 9: Sampling Distributions of Estimators

Topic 9: Sampling Distributions of Estimators Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be

More information

Lecture 23: Minimal sufficiency

Lecture 23: Minimal sufficiency Lecture 23: Miimal sufficiecy Maximal reductio without loss of iformatio There are may sufficiet statistics for a give problem. I fact, X (the whole data set) is sufficiet. If T is a sufficiet statistic

More information

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 3 9/11/2013. Large deviations Theory. Cramér s Theorem

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 3 9/11/2013. Large deviations Theory. Cramér s Theorem MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/5.070J Fall 203 Lecture 3 9//203 Large deviatios Theory. Cramér s Theorem Cotet.. Cramér s Theorem. 2. Rate fuctio ad properties. 3. Chage of measure techique.

More information

Hybrid Coding for Gaussian Broadcast Channels with Gaussian Sources

Hybrid Coding for Gaussian Broadcast Channels with Gaussian Sources Hybrid Codig for Gaussia Broadcast Chaels with Gaussia Sources Rajiv Soudararaja Departmet of Electrical & Computer Egieerig Uiversity of Texas at Austi Austi, TX 7871, USA Email: rajivs@mailutexasedu

More information