Quantitative Steganalysis of LSB Embedding in JPEG Domain

Size: px
Start display at page:

Download "Quantitative Steganalysis of LSB Embedding in JPEG Domain"

Transcription

1 Quantitative Steganalysis of LSB Embedding in JPEG Domain Jan Kodovský, Jessica Fridrich September 10, 2010 / ACM MM&Sec 10 1 / 17

2 Motivation Least Significant Bit (LSB) embedding Simplicity, high embedding capacity Used in Jsteg, JP Hide&Seek, and other commercial stego software Steganalysis of LSB embedding in spatial domain is mature area [Dumitrescu-2002], [Ker-2008] Our focus Transform domain JPEG format Quantitative steganalysis Outputs the estimate of the message length 2 / 17

3 Jsteg Jsteg: [Upham-1993] LSB replacement Embedding along a pseudo-random path DCT histogram Skipping 0 and / 17

4 Jsteg Jsteg: [Upham-1993] LSB replacement Embedding along a pseudo-random path Full embedding Skipping 0 and Embedding violates histogram symmetry 3 / 17

5 Selected Existing Attacks [Zhang,Ping-2003] the first quantitative attack Employed violation of histogram symmetry [Yu-2004] histogram-based attack Generalized Cauchy ML fit Chi-square test [Lee-2006], [Lee-2007] Category attack Technically not quantitative [Westfeld-2007], [Böhme-2008] adaptation of spatial domain attacks [Pevný-2009] support vector regression Feature-based non-structural attack Currently the most accurate quantitative attack 4 / 17

6 Our Goals / Challenges Improve the accuracy of existing quantitative attacks to Jsteg Achieve better performance than the feature-based machine learning approach (SVR) Focus on the structure of LSB embedding Deliver theoretically well-founded modular framework Explore the applicability of the proposed attacks to a different LSB embedding paradigms 5 / 17

7 Maximum Likelihood β... change rate P x (x) x Emb(β) Emb(β) P (y x,β) P (y x,β) cover feature vector x stego feature vector y P (y, β) = P (y, x, β)dx = ˆβ = arg max β 0 P (y x, β)p (x, β)dx = P (β) P (y β) = arg max β 0 P (y x, β)p x(x)dx Choice of the feature vector x is crucial P (y x, β)p x(x)dx 6 / 17

8 Features of Zhang & Ping x = [ x 1 x 2 x 3 ] [Zhang,Ping-2003] P (y x, β) Binomial distribution Gaussian approximation Embedding invariants: x 1 + x 2, x 3 P x (x) precover assumption [Ker-2007] 7 / 17

9 Features of Zhang & Ping x = [ x 1 x 2 x 3 ] [Zhang,Ping-2003] P (y x, β) Binomial distribution Gaussian approximation Embedding invariants: x 1 + x 2, x 3 P x (x) precover assumption [Ker-2007] 7 / 17

10 Features of Zhang & Ping x = [ x 1 x 2 x 3 ] [Zhang,Ping-2003] P (y x, β) Binomial distribution Gaussian approximation Embedding invariants: x 1 + x 2, x 3 P x (x) precover assumption [Ker-2007] 7 / 17

11 Features of Zhang & Ping x = [ x 1 x 2 x 3 ] [Zhang,Ping-2003] P (y x, β) Binomial distribution Gaussian approximation Embedding invariants: x 1 + x 2, x 3 P x (x) precover assumption [Ker-2007] 7 / 17

12 Features of Zhang & Ping Emb(β) x = [ x 1 x 2 x 3 ] β 1 β y = [ x β 1 x β 2 x β 3 ] [Zhang,Ping-2003] arg max β 0 P (y x, β)p x(x)dx P (y x, β) Binomial distribution Gaussian approximation Embedding invariants: x 1 + x 2, x 3 P x (x) precover assumption [Ker-2007] Precover 1/2 1/2 x 1 x 2 + x 3 7 / 17

13 Performance Evaluation Median absolute error 10 2 Jsteg ML - Zhang & Ping Change rate β 3,250 JPEG images resized and compressed to QF=75 Performance similar to [Zhang,Ping-2003] Assumption x β 1 = expected value Zhang & Ping s estimator 8 / 17

14 Performance Evaluation Median absolute error 10 2 Jsteg ML - Zhang & Ping SVR Cartesian-calibrated Pevný features (548) Additional 3,250 images for training Change rate β 3,250 JPEG images resized and compressed to QF=75 Performance similar to [Zhang,Ping-2003] Assumption x β 1 = expected value Zhang & Ping s estimator 8 / 17

15 First-Order Statistics x = [ x 2L, x 2L+1,..., x 2R, x 2R+1 ] ˆβ = arg max β 0 P (y x, β)p x(x)dx Embedding changes in individual LSB pairs are independent ( ) ( ) P (y x, β) = P x β 0 x 0, β P x β 1 x 1, β P Embedding invariants: x 0, x 1, x 2k + x 2k+1 Binomial distribution Gaussian approximation k ( ) x β 2k, xβ 2k+1 x 2k, x 2k+1, β 9 / 17

16 First-Order Statistics x = [ x 2L, x 2L+1,..., x 2R, x 2R+1 ] ˆβ = arg max β 0 P (y x, β)p x(x)dx p 1 2s ( x + 1 ) p s DCT coefficients are i.i.d. drawn from generalized Cauchy distribution Parameters p and s are ML estimates, given embedding invariants Precover assumption for every LSB pair Precover Embedding invariants: x 0, x 1, x 2k + x 2k+1 9 / 17

17 Performance Evaluation Median absolute error 10 2 Jsteg ML - Zhang & Ping ML - First-order SVR Change rate β 10 / 17

18 Second-Order Statistics DCT coefficients are not i.i.d. We capture dependencies using adjacency matrix X Natural decomposition into k-nodes, k {1, 2, 4} Binomial / multinomial distributions Gaussian approximations [-2,3] [-1,3] [0,3] [1,3] [2,3] [3,3] [-2,2] [-1,2] [0,2] [1,2] [2,2] [3,2] arg max β 0 P (Y X, β)p x(x)dx [-2,1] [-1,1] [0,1] [1,1] [2,1] [3,1] [-2,0] [-1,0] [0,0] [1,0] [2,0] [3,0] Factorization of P (y x, β) [-2,-1] [-1,-1] [0,-1] [1,-1] [2,-1] [3,-1] Embedding invariants [-2,-2] [-1,-2] [0,-2] [1,-2] [2,-2] [3,-2] Analytic expression 11 / 17

19 Second-Order Statistics DCT coefficients are not i.i.d. We capture dependencies using adjacency matrix X Natural decomposition into k-nodes, k {1, 2, 4} Binomial / multinomial distributions Gaussian approximations [-2,3] [-1,3] [0,3] [1,3] [2,3] [3,3] [-2,2] [-1,2] [0,2] [1,2] [2,2] [3,2] arg max β 0 P (Y X, β)p x(x)dx [-2,1] [-1,1] [0,1] [1,1] [2,1] [3,1] [-2,0] [-1,0] [0,0] [1,0] [2,0] [3,0] Complications arise [-2,-1] [-1,-1] [0,-1] [1,-1] [2,-1] [3,-1] Good parametric model? [-2,-2] [-1,-2] [0,-2] [1,-2] [2,-2] [3,-2] High complexity 11 / 17

20 Zero Message Hypothesis (ZMH) Alternative heuristic approach Penalty function z(x) 0 satisfying z(x β ) 0 when β = 0 z(x β ) > 0 when β > 0 z(x) should be a quantitative description of a zero message hypothesis capturing a key cover property violated by embedding Assumption: y = E[x β ] = Emb(x, β) Assumption: mapping Emb is invertible x = Emb 1 (y, β) Comments ˆβ = arg min β 0 z(emb 1 (y, β)) Low computational complexity one-dimensional search over β ZMH-based steganalysis is not a new idea! [RS steganalysis,2001] 12 / 17

21 First-Order Statistics (ZMH) x = [x 2L, x 2L+1,..., x 2R 1, x 2R ] Penalty function z sym (x) = w k (x k x k ) 2 Weights w k chosen to minimize the estimator variance least squares steganalysis [Ker-2007] Final form of the penalty function: z sym (x) = k>0 (x k x k ) 2 x k + x k 13 / 17

22 Performance Evaluation Median absolute error 10 2 Jsteg ML - Zhang & Ping ML - First-order ZMH - First-order SVR Change rate β 14 / 17

23 Second-Order Statistics (ZMH) Feature vector: adjacency matrix X ZMH approach Decomposition into k-nodes Embedding is invertible provided 0 β < 1/2 Symmetry about D [-2,3] [-2,2] [-1,3] [-1,2] [0,3] [0,2] [1,3] [1,2] [2,3] [2,2] [3,3] [3,2] [-2,1] [-1,1] [0,1] [1,1] [2,1] [3,1] z adj (X) = i,j (x i,j x j, i ) 2 x i,j + x j, i [-2,0] [-1,0] [0,0] [1,0] [2,0] [3,0] [-2,-1] [-1,-1] [0,-1] [1,-1] [2,-1] [3,-1] [-2,-2] [-1,-2] [0,-2] [1,-2] [2,-2] [3,-2] D 15 / 17

24 Second-Order Statistics (ZMH) Feature vector: adjacency matrix X ZMH approach Decomposition into k-nodes Embedding is invertible provided 0 β < 1/2 Symmetry about D [-2,3] [-2,2] [-1,3] [-1,2] [0,3] [0,2] [1,3] [1,2] [2,3] [2,2] [3,3] [3,2] [-2,1] [-1,1] [0,1] [1,1] [2,1] [3,1] z adj (X) = i,j (x i,j x j, i ) 2 x i,j + x j, i [-2,0] [-1,0] [0,0] [1,0] [2,0] [3,0] [-2,-1] [-1,-1] [0,-1] [1,-1] [2,-1] [3,-1] [-2,-2] [-1,-2] [0,-2] [1,-2] [2,-2] [3,-2] D 15 / 17

25 Performance Evaluation Median absolute error 10 2 Jsteg ML - Zhang & Ping ML - First-order ZMH - First-order ZMH - Second-order SVR Change rate β 16 / 17

26 What Else Can You Find in the Paper / Journal Version Error analysis of between-image and within-image errors for selected attacks Verification of precover assumptions using two different statistical tests Discussion & experiments with the symmetrized version of Jsteg Conversion of the Category attack [Lee-2006] into a quantitative one through the proposed ZMH framework Experiments conducted on two different sources of images Results reported in terms of two more security measures: IQR, median bias 17 / 17

Quantitative Structural Steganalysis of Jsteg

Quantitative Structural Steganalysis of Jsteg Quantitative Structural Steganalysis of Jsteg Jan Kodovský and Jessica Fridrich, Member, IEEE Abstract Quantitative steganalysis strives to estimate the change rate defined as the relative number of embedding

More information

Fisher Information Determines Capacity of ε-secure Steganography

Fisher Information Determines Capacity of ε-secure Steganography Fisher Information Determines Capacity of ε-secure Steganography Tomáš Filler and Jessica Fridrich Dept. of Electrical and Computer Engineering SUNY Binghamton, New York 11th Information Hiding, Darmstadt,

More information

Complete characterization of perfectly secure stego-systems with mutually independent embedding operation

Complete characterization of perfectly secure stego-systems with mutually independent embedding operation Complete characterization of perfectly secure stego-systems with mutually independent embedding operation Tomáš Filler and Jessica Fridrich Dept. of Electrical and Computer Engineering SUNY Binghamton,

More information

JPEG-Compatibility Steganalysis Using Block-Histogram of Recompression Artifacts

JPEG-Compatibility Steganalysis Using Block-Histogram of Recompression Artifacts JPEG-Compatibility Steganalysis Using Block-Histogram of Recompression Artifacts Jan Kodovský and Jessica Fridrich Department of ECE, Binghamton University, NY, USA {fridrich,jan.kodovsky}@binghamton.edu

More information

Generalized Transfer Component Analysis for Mismatched Jpeg Steganalysis

Generalized Transfer Component Analysis for Mismatched Jpeg Steganalysis Generalized Transfer Component Analysis for Mismatched Jpeg Steganalysis Xiaofeng Li 1, Xiangwei Kong 1, Bo Wang 1, Yanqing Guo 1,Xingang You 2 1 School of Information and Communication Engineering Dalian

More information

ROC Curves for Steganalysts

ROC Curves for Steganalysts ROC Curves for Steganalysts Andreas Westfeld Technische Universität Dresden Institut für Systemarchitektur 01062 Dresden mailto:westfeld@inf.tu-dresden.de 1 Introduction There are different approaches

More information

Image Data Compression. Steganography and steganalysis Alexey Pak, PhD, Lehrstuhl für Interak;ve Echtzeitsysteme, Fakultät für Informa;k, KIT

Image Data Compression. Steganography and steganalysis Alexey Pak, PhD, Lehrstuhl für Interak;ve Echtzeitsysteme, Fakultät für Informa;k, KIT Image Data Compression Steganography and steganalysis 1 Stenography vs watermarking Watermarking: impercep'bly altering a Work to embed a message about that Work Steganography: undetectably altering a

More information

The Square Root Law of Steganographic Capacity

The Square Root Law of Steganographic Capacity The Square Root Law of Steganographic Capacity ABSTRACT Andrew D. Ker Oxford University Computing Laboratory Parks Road Oxford OX1 3QD, UK adk@comlab.ox.ac.uk Jan Kodovský Dept. of Electrical and Computer

More information

Gaussian Processes. Le Song. Machine Learning II: Advanced Topics CSE 8803ML, Spring 2012

Gaussian Processes. Le Song. Machine Learning II: Advanced Topics CSE 8803ML, Spring 2012 Gaussian Processes Le Song Machine Learning II: Advanced Topics CSE 8803ML, Spring 01 Pictorial view of embedding distribution Transform the entire distribution to expected features Feature space Feature

More information

Information Hiding and Covert Communication

Information Hiding and Covert Communication Information Hiding and Covert Communication Andrew Ker adk @ comlab.ox.ac.uk Royal Society University Research Fellow Oxford University Computing Laboratory Foundations of Security Analysis and Design

More information

Optimal Detection of OutGuess using an Accurate Model of DCT Coefficients

Optimal Detection of OutGuess using an Accurate Model of DCT Coefficients Optimal Detection of OutGuess using an Accurate Model of DCT Coefficients Thanh Hai Thai, Rémi Cogranne, Florent Retraint ICD - ROSAS - LMS - Troyes University of Technology - UMR 68, CNRS Troyes - France

More information

Theoretical Model of the FLD Ensemble Classifier Based on Hypothesis Testing Theory

Theoretical Model of the FLD Ensemble Classifier Based on Hypothesis Testing Theory Theoretical Model of the FLD Ensemble Classifier Based on Hypothesis Testing Theory Rémi Cogranne, Member, IEEE, ICD - ROSAS - LM2S Troyes University of Technology - UMR 6281, CNRS 12, rue Marie Curie

More information

Detection of Content Adaptive LSB Matching (a Game Theory Approach)

Detection of Content Adaptive LSB Matching (a Game Theory Approach) Detection of Content Adaptive LSB Matching (a Game Theory Approach) Tomáš Denemark and Jessica Fridrich Department of ECE, SUNY Binghamton, NY, USA ABSTRACT This paper is an attempt to analyze the interaction

More information

Machine Learning Basics: Maximum Likelihood Estimation

Machine Learning Basics: Maximum Likelihood Estimation Machine Learning Basics: Maximum Likelihood Estimation Sargur N. srihari@cedar.buffalo.edu This is part of lecture slides on Deep Learning: http://www.cedar.buffalo.edu/~srihari/cse676 1 Topics 1. Learning

More information

Machine Learning. Lecture 4: Regularization and Bayesian Statistics. Feng Li. https://funglee.github.io

Machine Learning. Lecture 4: Regularization and Bayesian Statistics. Feng Li. https://funglee.github.io Machine Learning Lecture 4: Regularization and Bayesian Statistics Feng Li fli@sdu.edu.cn https://funglee.github.io School of Computer Science and Technology Shandong University Fall 207 Overfitting Problem

More information

Detection of Content Adaptive LSB Matching (a Game Theory Approach)

Detection of Content Adaptive LSB Matching (a Game Theory Approach) Detection of Content Adaptive LSB Matching (a Game Theory Approach) Tomáš Denemark and Jessica Fridrich Department of ECE, SUNY Binghamton, NY, USA ABSTRACT This paper is an attempt to analyze the interaction

More information

Learning outcomes. Palettes and GIF. The colour palette. Using the colour palette The GIF file. CSM25 Secure Information Hiding

Learning outcomes. Palettes and GIF. The colour palette. Using the colour palette The GIF file. CSM25 Secure Information Hiding Learning outcomes Palettes and GIF CSM25 Secure Information Hiding Dr Hans Georg Schaathun University of Surrey Learn how images are represented using a palette Get an overview of hiding techniques in

More information

Improved Adaptive LSB Steganography based on Chaos and Genetic Algorithm

Improved Adaptive LSB Steganography based on Chaos and Genetic Algorithm Improved Adaptive LSB Steganography based on Chaos and Genetic Algorithm Lifang Yu, Yao Zhao 1, Rongrong Ni, Ting Li Institute of Information Science, Beijing Jiaotong University, BJ 100044, China Abstract

More information

Lecture 3: Statistical Decision Theory (Part II)

Lecture 3: Statistical Decision Theory (Part II) Lecture 3: Statistical Decision Theory (Part II) Hao Helen Zhang Hao Helen Zhang Lecture 3: Statistical Decision Theory (Part II) 1 / 27 Outline of This Note Part I: Statistics Decision Theory (Classical

More information

2. Definition & Classification

2. Definition & Classification Information Hiding Data Hiding K-H Jung Agenda 1. Data Hiding 2. Definition & Classification 3. Related Works 4. Considerations Definition of Data Hiding 3 Data Hiding, Information Hiding Concealing secret

More information

Steganalysis of Spread Spectrum Data Hiding Exploiting Cover Memory

Steganalysis of Spread Spectrum Data Hiding Exploiting Cover Memory Steganalysis of Spread Spectrum Data Hiding Exploiting Cover Memory Kenneth Sullivan, Upamanyu Madhow, Shivkumar Chandrasekaran, and B.S. Manjunath Department of Electrical and Computer Engineering University

More information

Steganalysis with a Computational Immune System

Steganalysis with a Computational Immune System DIGITAL FORENSIC RESEARCH CONFERENCE Steganalysis with a Computational Immune System By Jacob Jackson, Gregg Gunsch, Roger Claypoole, Gary Lamont Presented At The Digital Forensic Research Conference DFRWS

More information

MINIMIZING ADDITIVE DISTORTION FUNCTIONS WITH NON-BINARY EMBEDDING OPERATION IN STEGANOGRAPHY. Tomáš Filler and Jessica Fridrich

MINIMIZING ADDITIVE DISTORTION FUNCTIONS WITH NON-BINARY EMBEDDING OPERATION IN STEGANOGRAPHY. Tomáš Filler and Jessica Fridrich MINIMIZING ADDITIVE DISTORTION FUNCTIONS WITH NON-BINARY EMBEDDING OPERATION IN STEGANOGRAPHY Tomáš Filler and Jessica Fridrich Department of ECE, SUNY Binghamton, NY, USA {tomas.filler, fridrich}@binghamton.edu

More information

Improving Selection-Channel-Aware Steganalysis Features

Improving Selection-Channel-Aware Steganalysis Features Improving Selection-Channel-Aware Steganalysis Features Tomáš Denemark and Jessica Fridrich, Department of ECE, SUNY Binghamton, NY, USA, {tdenema1,fridrich}@binghamton.edu, Pedro Comesaña-Alfaro, Department

More information

PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 2: PROBABILITY DISTRIBUTIONS

PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 2: PROBABILITY DISTRIBUTIONS PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 2: PROBABILITY DISTRIBUTIONS Parametric Distributions Basic building blocks: Need to determine given Representation: or? Recall Curve Fitting Binary Variables

More information

6.867 Machine Learning

6.867 Machine Learning 6.867 Machine Learning Problem set 1 Solutions Thursday, September 19 What and how to turn in? Turn in short written answers to the questions explicitly stated, and when requested to explain or prove.

More information

A Bayesian Perspective on Residential Demand Response Using Smart Meter Data

A Bayesian Perspective on Residential Demand Response Using Smart Meter Data A Bayesian Perspective on Residential Demand Response Using Smart Meter Data Datong-Paul Zhou, Maximilian Balandat, and Claire Tomlin University of California, Berkeley [datong.zhou, balandat, tomlin]@eecs.berkeley.edu

More information

Image Dependent Log-likelihood Ratio Allocation for Repeat Accumulate Code based Decoding in Data Hiding Channels

Image Dependent Log-likelihood Ratio Allocation for Repeat Accumulate Code based Decoding in Data Hiding Channels Image Dependent Log-likelihood Ratio Allocation for Repeat Accumulate Code based Decoding in Data Hiding Channels Anindya Sarkar and B. S. Manjunath Department of Electrical and Computer Engineering, University

More information

Imperfect Stegosystems

Imperfect Stegosystems DISSERTATION DEFENSE Imperfect Stegosystems Asymptotic Laws and Near-Optimal Practical Constructions Tomá² Filler Dept. of Electrical and Computer Engineering SUNY Binghamton, New York April 1, 2011 2

More information

USING MULTIPLE RE-EMBEDDINGS FOR QUANTITATIVE STEGANALYSIS AND IMAGE RELIABILITY ESTIMATION

USING MULTIPLE RE-EMBEDDINGS FOR QUANTITATIVE STEGANALYSIS AND IMAGE RELIABILITY ESTIMATION TKK Reports in Information and Computer Science Espoo 2010 TKK-ICS-R34 USING MULTIPLE RE-EMBEDDINGS FOR QUANTITATIVE STEGANALYSIS AND IMAGE RELIABILITY ESTIMATION Yoan Miche, Patrick Bas and Amaury Lendasse

More information

6.867 Machine Learning

6.867 Machine Learning 6.867 Machine Learning Problem set 1 Due Thursday, September 19, in class What and how to turn in? Turn in short written answers to the questions explicitly stated, and when requested to explain or prove.

More information

Hypothesis testing:power, test statistic CMS:

Hypothesis testing:power, test statistic CMS: Hypothesis testing:power, test statistic The more sensitive the test, the better it can discriminate between the null and the alternative hypothesis, quantitatively, maximal power In order to achieve this

More information

IEOR 165 Lecture 7 1 Bias-Variance Tradeoff

IEOR 165 Lecture 7 1 Bias-Variance Tradeoff IEOR 165 Lecture 7 Bias-Variance Tradeoff 1 Bias-Variance Tradeoff Consider the case of parametric regression with β R, and suppose we would like to analyze the error of the estimate ˆβ in comparison to

More information

Detection of Content Adaptive LSB Matching (a Game Theory Approach)

Detection of Content Adaptive LSB Matching (a Game Theory Approach) Detection of Content Adaptive LSB Matching (a Game Theory Approach) Tomáš Denemark, Jessica Fridrich / 5 Content-adaptive steganography Every pixel is changed with probability β i = exp( λρ i) + exp( λρ

More information

Low Complexity Features for JPEG Steganalysis Using Undecimated DCT

Low Complexity Features for JPEG Steganalysis Using Undecimated DCT 1 Low Complexity Features for JPEG Steganalysis Using Undecimated DCT Vojtěch Holub and Jessica Fridrich, Member, IEEE Abstract This article introduces a novel feature set for steganalysis of JPEG images

More information

Research Article Improved Adaptive LSB Steganography Based on Chaos and Genetic Algorithm

Research Article Improved Adaptive LSB Steganography Based on Chaos and Genetic Algorithm Hindawi Publishing Corporation EURASIP Journal on Advances in Signal Processing Volume 2010, Article ID 876946, 6 pages doi:10.1155/2010/876946 Research Article Improved Adaptive LSB Steganography Based

More information

DIGITAL image processing has remarkably developed

DIGITAL image processing has remarkably developed IEEE TRANSACTION ON IMAGE PROCESSING SUBMITTED VERSION, OCT. 3 Statistical Model of Quantized DCT Coefficients : Application in the Steganalysis of Jsteg Algorithm Thanh Hai Thai, Rémi Cogranne, Florent

More information

Machine Learning. Linear Models. Fabio Vandin October 10, 2017

Machine Learning. Linear Models. Fabio Vandin October 10, 2017 Machine Learning Linear Models Fabio Vandin October 10, 2017 1 Linear Predictors and Affine Functions Consider X = R d Affine functions: L d = {h w,b : w R d, b R} where ( d ) h w,b (x) = w, x + b = w

More information

Accouncements. You should turn in a PDF and a python file(s) Figure for problem 9 should be in the PDF

Accouncements. You should turn in a PDF and a python file(s) Figure for problem 9 should be in the PDF Accouncements You should turn in a PDF and a python file(s) Figure for problem 9 should be in the PDF Please do not zip these files and submit (unless there are >5 files) 1 Bayesian Methods Machine Learning

More information

CONTENT ADAPTIVE STEGANOGRAPHY DESIGN AND DETECTION

CONTENT ADAPTIVE STEGANOGRAPHY DESIGN AND DETECTION CONTENT ADAPTIVE STEGANOGRAPHY DESIGN AND DETECTION BY VOJTĚCH HOLUB MS, Czech Technical University, Prague, 2010 DISSERTATION Submitted in partial fulfillment of the requirements for the degree of Doctor

More information

Obtaining Higher Rates for Steganographic Schemes while Maintaining the Same Detectability

Obtaining Higher Rates for Steganographic Schemes while Maintaining the Same Detectability Obtaining Higher Rates for Steganographic Schemes while Maintaining the Same Detectability Anindya Sarkar, Kaushal Solanki and and B. S. Manjunath Department of Electrical and Computer Engineering, University

More information

Introduction to Statistical Learning Theory

Introduction to Statistical Learning Theory Introduction to Statistical Learning Theory In the last unit we looked at regularization - adding a w 2 penalty. We add a bias - we prefer classifiers with low norm. How to incorporate more complicated

More information

Lecture 5. Ideal and Almost-Ideal SG.

Lecture 5. Ideal and Almost-Ideal SG. Lecture 5. Ideal and Almost-Ideal SG. Definition. SG is called ideal (perfect or unconditionally undetectable ISG),if its detection is equivalently to random guessing of this fact even with the use of

More information

Matrix Embedding with Pseudorandom Coefficient Selection and Error Correction for Robust and Secure Steganography

Matrix Embedding with Pseudorandom Coefficient Selection and Error Correction for Robust and Secure Steganography Matrix Embedding with Pseudorandom Coefficient Selection and Error Correction for Robust and Secure Steganography Anindya Sarkar, Student Member, IEEE, Upamanyu Madhow, Fellow, IEEE, and B. S. Manjunath,

More information

Lecture 4: Types of errors. Bayesian regression models. Logistic regression

Lecture 4: Types of errors. Bayesian regression models. Logistic regression Lecture 4: Types of errors. Bayesian regression models. Logistic regression A Bayesian interpretation of regularization Bayesian vs maximum likelihood fitting more generally COMP-652 and ECSE-68, Lecture

More information

Steganalysis of ±k Steganography based on Noncausal Linear Predictor

Steganalysis of ±k Steganography based on Noncausal Linear Predictor INTERNATIONAL JOURNAL OF COMPUTERS COMMUNICATIONS & CONTROL ISSN 1841-9836, 9(5):623-632, October, 2014. Steganalysis of ±k Steganography based on Noncausal Linear Predictor K.M. Singh, Y.J. Chanu, T.

More information

Sparse Linear Models (10/7/13)

Sparse Linear Models (10/7/13) STA56: Probabilistic machine learning Sparse Linear Models (0/7/) Lecturer: Barbara Engelhardt Scribes: Jiaji Huang, Xin Jiang, Albert Oh Sparsity Sparsity has been a hot topic in statistics and machine

More information

Lecture : Probabilistic Machine Learning

Lecture : Probabilistic Machine Learning Lecture : Probabilistic Machine Learning Riashat Islam Reasoning and Learning Lab McGill University September 11, 2018 ML : Many Methods with Many Links Modelling Views of Machine Learning Machine Learning

More information

Parametric Models. Dr. Shuang LIANG. School of Software Engineering TongJi University Fall, 2012

Parametric Models. Dr. Shuang LIANG. School of Software Engineering TongJi University Fall, 2012 Parametric Models Dr. Shuang LIANG School of Software Engineering TongJi University Fall, 2012 Today s Topics Maximum Likelihood Estimation Bayesian Density Estimation Today s Topics Maximum Likelihood

More information

Naïve Bayes classification

Naïve Bayes classification Naïve Bayes classification 1 Probability theory Random variable: a variable whose possible values are numerical outcomes of a random phenomenon. Examples: A person s height, the outcome of a coin toss

More information

COMS 4771 Regression. Nakul Verma

COMS 4771 Regression. Nakul Verma COMS 4771 Regression Nakul Verma Last time Support Vector Machines Maximum Margin formulation Constrained Optimization Lagrange Duality Theory Convex Optimization SVM dual and Interpretation How get the

More information

A Reservoir Sampling Algorithm with Adaptive Estimation of Conditional Expectation

A Reservoir Sampling Algorithm with Adaptive Estimation of Conditional Expectation A Reservoir Sampling Algorithm with Adaptive Estimation of Conditional Expectation Vu Malbasa and Slobodan Vucetic Abstract Resource-constrained data mining introduces many constraints when learning from

More information

Perturbed Quantization Steganography

Perturbed Quantization Steganography Perturbed Quantization Steganography Jessica Fridrich SUNY Binghamton Department of ECE Binghamton, NY 13902-6000 001 607 777 2577 fridrich@binghamton.edu Miroslav Goljan SUNY Binghamton Department of

More information

Linear Models for Regression CS534

Linear Models for Regression CS534 Linear Models for Regression CS534 Prediction Problems Predict housing price based on House size, lot size, Location, # of rooms Predict stock price based on Price history of the past month Predict the

More information

New Steganographic scheme based of Reed- Solomon codes

New Steganographic scheme based of Reed- Solomon codes New Steganographic scheme based of Reed- Solomon codes I. DIOP; S.M FARSSI ;O. KHOUMA ; H. B DIOUF ; K.TALL ; K.SYLLA Ecole Supérieure Polytechnique de l Université Dakar Sénégal Email: idydiop@yahoo.fr;

More information

Regression I: Mean Squared Error and Measuring Quality of Fit

Regression I: Mean Squared Error and Measuring Quality of Fit Regression I: Mean Squared Error and Measuring Quality of Fit -Applied Multivariate Analysis- Lecturer: Darren Homrighausen, PhD 1 The Setup Suppose there is a scientific problem we are interested in solving

More information

A Study of Embedding Operations and Locations for Steganography in H.264 Video

A Study of Embedding Operations and Locations for Steganography in H.264 Video A Study of Embedding Operations and Locations for Steganography in H.264 Video Andreas Neufeld and Andrew D. Ker Oxford University Department of Computer Science Andreas Neufeld is now with the Image and

More information

STATISTICAL DETECTION OF INFORMATION HIDING BASED ON ADJACENT PIXELS DIFFERENCE

STATISTICAL DETECTION OF INFORMATION HIDING BASED ON ADJACENT PIXELS DIFFERENCE 2th European Signal Processing Conference EUSIPCO 22) Bucharest, Romania, August 27-3, 22 STATISTICAL DETECTION OF INFORMATION HIDING BASED ON ADJACENT PIXELS DIFFERENCE Rémi Cogranne, Cathel Zitzmann,

More information

Lecture 3: More on regularization. Bayesian vs maximum likelihood learning

Lecture 3: More on regularization. Bayesian vs maximum likelihood learning Lecture 3: More on regularization. Bayesian vs maximum likelihood learning L2 and L1 regularization for linear estimators A Bayesian interpretation of regularization Bayesian vs maximum likelihood fitting

More information

Introduction to Machine Learning. Lecture 2

Introduction to Machine Learning. Lecture 2 Introduction to Machine Learning Lecturer: Eran Halperin Lecture 2 Fall Semester Scribe: Yishay Mansour Some of the material was not presented in class (and is marked with a side line) and is given for

More information

Detecting LSB Matching by Applying Calibration Technique for Difference Image

Detecting LSB Matching by Applying Calibration Technique for Difference Image Detecting LSB atching by Applying Calibration Technique for Difference Image iaolong Li Institute of Computer Science & Technology Peking University 87, Beijing, P. R. China lixiaolong@icst.pku.edu.cn

More information

Data Mining Stat 588

Data Mining Stat 588 Data Mining Stat 588 Lecture 02: Linear Methods for Regression Department of Statistics & Biostatistics Rutgers University September 13 2011 Regression Problem Quantitative generic output variable Y. Generic

More information

Review. DS GA 1002 Statistical and Mathematical Models. Carlos Fernandez-Granda

Review. DS GA 1002 Statistical and Mathematical Models.   Carlos Fernandez-Granda Review DS GA 1002 Statistical and Mathematical Models http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall16 Carlos Fernandez-Granda Probability and statistics Probability: Framework for dealing with

More information

EXAM IN STATISTICAL MACHINE LEARNING STATISTISK MASKININLÄRNING

EXAM IN STATISTICAL MACHINE LEARNING STATISTISK MASKININLÄRNING EXAM IN STATISTICAL MACHINE LEARNING STATISTISK MASKININLÄRNING DATE AND TIME: August 30, 2018, 14.00 19.00 RESPONSIBLE TEACHER: Niklas Wahlström NUMBER OF PROBLEMS: 5 AIDING MATERIAL: Calculator, mathematical

More information

High-dimensional regression

High-dimensional regression High-dimensional regression Advanced Methods for Data Analysis 36-402/36-608) Spring 2014 1 Back to linear regression 1.1 Shortcomings Suppose that we are given outcome measurements y 1,... y n R, and

More information

Probabilistic modeling. The slides are closely adapted from Subhransu Maji s slides

Probabilistic modeling. The slides are closely adapted from Subhransu Maji s slides Probabilistic modeling The slides are closely adapted from Subhransu Maji s slides Overview So far the models and algorithms you have learned about are relatively disconnected Probabilistic modeling framework

More information

Naïve Bayes classification. p ij 11/15/16. Probability theory. Probability theory. Probability theory. X P (X = x i )=1 i. Marginal Probability

Naïve Bayes classification. p ij 11/15/16. Probability theory. Probability theory. Probability theory. X P (X = x i )=1 i. Marginal Probability Probability theory Naïve Bayes classification Random variable: a variable whose possible values are numerical outcomes of a random phenomenon. s: A person s height, the outcome of a coin toss Distinguish

More information

Support Vector Machine I

Support Vector Machine I Support Vector Machine I Jia-Bin Huang ECE-5424G / CS-5824 Virginia Tech Spring 2019 Administrative Please use piazza. No emails. HW 0 grades are back. Re-grade request for one week. HW 1 due soon. HW

More information

STATISTICS SYLLABUS UNIT I

STATISTICS SYLLABUS UNIT I STATISTICS SYLLABUS UNIT I (Probability Theory) Definition Classical and axiomatic approaches.laws of total and compound probability, conditional probability, Bayes Theorem. Random variable and its distribution

More information

Machine Learning CSE546 Carlos Guestrin University of Washington. September 30, 2013

Machine Learning CSE546 Carlos Guestrin University of Washington. September 30, 2013 Bayesian Methods Machine Learning CSE546 Carlos Guestrin University of Washington September 30, 2013 1 What about prior n Billionaire says: Wait, I know that the thumbtack is close to 50-50. What can you

More information

Introduction to Statistical modeling: handout for Math 489/583

Introduction to Statistical modeling: handout for Math 489/583 Introduction to Statistical modeling: handout for Math 489/583 Statistical modeling occurs when we are trying to model some data using statistical tools. From the start, we recognize that no model is perfect

More information

Modern Methods of Data Analysis - WS 07/08

Modern Methods of Data Analysis - WS 07/08 Modern Methods of Data Analysis Lecture VII (26.11.07) Contents: Maximum Likelihood (II) Exercise: Quality of Estimators Assume hight of students is Gaussian distributed. You measure the size of N students.

More information

STA414/2104 Statistical Methods for Machine Learning II

STA414/2104 Statistical Methods for Machine Learning II STA414/2104 Statistical Methods for Machine Learning II Murat A. Erdogdu & David Duvenaud Department of Computer Science Department of Statistical Sciences Lecture 3 Slide credits: Russ Salakhutdinov Announcements

More information

Statistical learning. Chapter 20, Sections 1 3 1

Statistical learning. Chapter 20, Sections 1 3 1 Statistical learning Chapter 20, Sections 1 3 Chapter 20, Sections 1 3 1 Outline Bayesian learning Maximum a posteriori and maximum likelihood learning Bayes net learning ML parameter learning with complete

More information

3 Joint Distributions 71

3 Joint Distributions 71 2.2.3 The Normal Distribution 54 2.2.4 The Beta Density 58 2.3 Functions of a Random Variable 58 2.4 Concluding Remarks 64 2.5 Problems 64 3 Joint Distributions 71 3.1 Introduction 71 3.2 Discrete Random

More information

Reminders. Thought questions should be submitted on eclass. Please list the section related to the thought question

Reminders. Thought questions should be submitted on eclass. Please list the section related to the thought question Linear regression Reminders Thought questions should be submitted on eclass Please list the section related to the thought question If it is a more general, open-ended question not exactly related to a

More information

Lecture 7 Introduction to Statistical Decision Theory

Lecture 7 Introduction to Statistical Decision Theory Lecture 7 Introduction to Statistical Decision Theory I-Hsiang Wang Department of Electrical Engineering National Taiwan University ihwang@ntu.edu.tw December 20, 2016 1 / 55 I-Hsiang Wang IT Lecture 7

More information

Information Hiding. Andrew D. Ker. 10 Lectures, Hilary Term (complete) Department of Computer Science, Oxford University

Information Hiding. Andrew D. Ker. 10 Lectures, Hilary Term (complete) Department of Computer Science, Oxford University Information Hiding (complete) Andrew D. Ker 10 Lectures, Hilary Term 2016 Department of Computer Science, Oxford University ii Contents 0 Introduction 1 0.1 Preliminary Material..............................

More information

Gaussians Linear Regression Bias-Variance Tradeoff

Gaussians Linear Regression Bias-Variance Tradeoff Readings listed in class website Gaussians Linear Regression Bias-Variance Tradeoff Machine Learning 10701/15781 Carlos Guestrin Carnegie Mellon University January 22 nd, 2007 Maximum Likelihood Estimation

More information

CSC2515 Winter 2015 Introduction to Machine Learning. Lecture 2: Linear regression

CSC2515 Winter 2015 Introduction to Machine Learning. Lecture 2: Linear regression CSC2515 Winter 2015 Introduction to Machine Learning Lecture 2: Linear regression All lecture slides will be available as.pdf on the course website: http://www.cs.toronto.edu/~urtasun/courses/csc2515/csc2515_winter15.html

More information

DS-GA 1002 Lecture notes 11 Fall Bayesian statistics

DS-GA 1002 Lecture notes 11 Fall Bayesian statistics DS-GA 100 Lecture notes 11 Fall 016 Bayesian statistics In the frequentist paradigm we model the data as realizations from a distribution that depends on deterministic parameters. In contrast, in Bayesian

More information

Confidence Intervals for Low-dimensional Parameters with High-dimensional Data

Confidence Intervals for Low-dimensional Parameters with High-dimensional Data Confidence Intervals for Low-dimensional Parameters with High-dimensional Data Cun-Hui Zhang and Stephanie S. Zhang Rutgers University and Columbia University September 14, 2012 Outline Introduction Methodology

More information

Pattern Recognition. Parameter Estimation of Probability Density Functions

Pattern Recognition. Parameter Estimation of Probability Density Functions Pattern Recognition Parameter Estimation of Probability Density Functions Classification Problem (Review) The classification problem is to assign an arbitrary feature vector x F to one of c classes. The

More information

Machine Learning. Gaussian Mixture Models. Zhiyao Duan & Bryan Pardo, Machine Learning: EECS 349 Fall

Machine Learning. Gaussian Mixture Models. Zhiyao Duan & Bryan Pardo, Machine Learning: EECS 349 Fall Machine Learning Gaussian Mixture Models Zhiyao Duan & Bryan Pardo, Machine Learning: EECS 349 Fall 2012 1 The Generative Model POV We think of the data as being generated from some process. We assume

More information

Linear Regression. CSL603 - Fall 2017 Narayanan C Krishnan

Linear Regression. CSL603 - Fall 2017 Narayanan C Krishnan Linear Regression CSL603 - Fall 2017 Narayanan C Krishnan ckn@iitrpr.ac.in Outline Univariate regression Multivariate regression Probabilistic view of regression Loss functions Bias-Variance analysis Regularization

More information

Linear Regression. CSL465/603 - Fall 2016 Narayanan C Krishnan

Linear Regression. CSL465/603 - Fall 2016 Narayanan C Krishnan Linear Regression CSL465/603 - Fall 2016 Narayanan C Krishnan ckn@iitrpr.ac.in Outline Univariate regression Multivariate regression Probabilistic view of regression Loss functions Bias-Variance analysis

More information

Motivation Scale Mixutres of Normals Finite Gaussian Mixtures Skew-Normal Models. Mixture Models. Econ 690. Purdue University

Motivation Scale Mixutres of Normals Finite Gaussian Mixtures Skew-Normal Models. Mixture Models. Econ 690. Purdue University Econ 690 Purdue University In virtually all of the previous lectures, our models have made use of normality assumptions. From a computational point of view, the reason for this assumption is clear: combined

More information

Sparse regression. Optimization-Based Data Analysis. Carlos Fernandez-Granda

Sparse regression. Optimization-Based Data Analysis.   Carlos Fernandez-Granda Sparse regression Optimization-Based Data Analysis http://www.cims.nyu.edu/~cfgranda/pages/obda_spring16 Carlos Fernandez-Granda 3/28/2016 Regression Least-squares regression Example: Global warming Logistic

More information

Approximate Kernel PCA with Random Features

Approximate Kernel PCA with Random Features Approximate Kernel PCA with Random Features (Computational vs. Statistical Tradeoff) Bharath K. Sriperumbudur Department of Statistics, Pennsylvania State University Journées de Statistique Paris May 28,

More information

Minimizing Embedding Impact in Steganography using Trellis-Coded Quantization

Minimizing Embedding Impact in Steganography using Trellis-Coded Quantization Minimizing Embedding Impact in Steganography using Trellis-Coded Quantization Tomáš Filler, Jan Judas, and Jessica Fridrich Department of Electrical and Computer Engineering SUNY Binghamton, Binghamton,

More information

Lecture 5: Logistic Regression. Neural Networks

Lecture 5: Logistic Regression. Neural Networks Lecture 5: Logistic Regression. Neural Networks Logistic regression Comparison with generative models Feed-forward neural networks Backpropagation Tricks for training neural networks COMP-652, Lecture

More information

Double Gamma Principal Components Analysis

Double Gamma Principal Components Analysis Applied Mathematical Sciences, Vol. 12, 2018, no. 11, 523-533 HIKARI Ltd, www.m-hikari.com https://doi.org/10.12988/ams.2018.8455 Double Gamma Principal Components Analysis Ameerah O. Bahashwan, Zakiah

More information

Statistical Machine Learning from Data

Statistical Machine Learning from Data Samy Bengio Statistical Machine Learning from Data 1 Statistical Machine Learning from Data Ensembles Samy Bengio IDIAP Research Institute, Martigny, Switzerland, and Ecole Polytechnique Fédérale de Lausanne

More information

STA 414/2104: Machine Learning

STA 414/2104: Machine Learning STA 414/2104: Machine Learning Russ Salakhutdinov Department of Computer Science! Department of Statistics! rsalakhu@cs.toronto.edu! http://www.cs.toronto.edu/~rsalakhu/ Lecture 9 Sequential Data So far

More information

Bayesian Machine Learning

Bayesian Machine Learning Bayesian Machine Learning Andrew Gordon Wilson ORIE 6741 Lecture 4 Occam s Razor, Model Construction, and Directed Graphical Models https://people.orie.cornell.edu/andrew/orie6741 Cornell University September

More information

Lecture 2 Machine Learning Review

Lecture 2 Machine Learning Review Lecture 2 Machine Learning Review CMSC 35246: Deep Learning Shubhendu Trivedi & Risi Kondor University of Chicago March 29, 2017 Things we will look at today Formal Setup for Supervised Learning Things

More information

Steganalysis in JPEG

Steganalysis in JPEG Steganalysis in JPEG CSM25 Secure Information Hiding Dr Hans Georg Schaathun University of Surrey Spring 2007 Dr Hans Georg Schaathun Steganalysis in JPEG Spring 2007 1 / 24 Learning Outcomes learn additional

More information

Bayesian Approach 2. CSC412 Probabilistic Learning & Reasoning

Bayesian Approach 2. CSC412 Probabilistic Learning & Reasoning CSC412 Probabilistic Learning & Reasoning Lecture 12: Bayesian Parameter Estimation February 27, 2006 Sam Roweis Bayesian Approach 2 The Bayesian programme (after Rev. Thomas Bayes) treats all unnown quantities

More information

Compressed Sensing and Linear Codes over Real Numbers

Compressed Sensing and Linear Codes over Real Numbers Compressed Sensing and Linear Codes over Real Numbers Henry D. Pfister (joint with Fan Zhang) Texas A&M University College Station Information Theory and Applications Workshop UC San Diego January 31st,

More information

Multiple regression. CM226: Machine Learning for Bioinformatics. Fall Sriram Sankararaman Acknowledgments: Fei Sha, Ameet Talwalkar

Multiple regression. CM226: Machine Learning for Bioinformatics. Fall Sriram Sankararaman Acknowledgments: Fei Sha, Ameet Talwalkar Multiple regression CM226: Machine Learning for Bioinformatics. Fall 2016 Sriram Sankararaman Acknowledgments: Fei Sha, Ameet Talwalkar Multiple regression 1 / 36 Previous two lectures Linear and logistic

More information