Quantitative Steganalysis of LSB Embedding in JPEG Domain
|
|
- Bernice Pitts
- 5 years ago
- Views:
Transcription
1 Quantitative Steganalysis of LSB Embedding in JPEG Domain Jan Kodovský, Jessica Fridrich September 10, 2010 / ACM MM&Sec 10 1 / 17
2 Motivation Least Significant Bit (LSB) embedding Simplicity, high embedding capacity Used in Jsteg, JP Hide&Seek, and other commercial stego software Steganalysis of LSB embedding in spatial domain is mature area [Dumitrescu-2002], [Ker-2008] Our focus Transform domain JPEG format Quantitative steganalysis Outputs the estimate of the message length 2 / 17
3 Jsteg Jsteg: [Upham-1993] LSB replacement Embedding along a pseudo-random path DCT histogram Skipping 0 and / 17
4 Jsteg Jsteg: [Upham-1993] LSB replacement Embedding along a pseudo-random path Full embedding Skipping 0 and Embedding violates histogram symmetry 3 / 17
5 Selected Existing Attacks [Zhang,Ping-2003] the first quantitative attack Employed violation of histogram symmetry [Yu-2004] histogram-based attack Generalized Cauchy ML fit Chi-square test [Lee-2006], [Lee-2007] Category attack Technically not quantitative [Westfeld-2007], [Böhme-2008] adaptation of spatial domain attacks [Pevný-2009] support vector regression Feature-based non-structural attack Currently the most accurate quantitative attack 4 / 17
6 Our Goals / Challenges Improve the accuracy of existing quantitative attacks to Jsteg Achieve better performance than the feature-based machine learning approach (SVR) Focus on the structure of LSB embedding Deliver theoretically well-founded modular framework Explore the applicability of the proposed attacks to a different LSB embedding paradigms 5 / 17
7 Maximum Likelihood β... change rate P x (x) x Emb(β) Emb(β) P (y x,β) P (y x,β) cover feature vector x stego feature vector y P (y, β) = P (y, x, β)dx = ˆβ = arg max β 0 P (y x, β)p (x, β)dx = P (β) P (y β) = arg max β 0 P (y x, β)p x(x)dx Choice of the feature vector x is crucial P (y x, β)p x(x)dx 6 / 17
8 Features of Zhang & Ping x = [ x 1 x 2 x 3 ] [Zhang,Ping-2003] P (y x, β) Binomial distribution Gaussian approximation Embedding invariants: x 1 + x 2, x 3 P x (x) precover assumption [Ker-2007] 7 / 17
9 Features of Zhang & Ping x = [ x 1 x 2 x 3 ] [Zhang,Ping-2003] P (y x, β) Binomial distribution Gaussian approximation Embedding invariants: x 1 + x 2, x 3 P x (x) precover assumption [Ker-2007] 7 / 17
10 Features of Zhang & Ping x = [ x 1 x 2 x 3 ] [Zhang,Ping-2003] P (y x, β) Binomial distribution Gaussian approximation Embedding invariants: x 1 + x 2, x 3 P x (x) precover assumption [Ker-2007] 7 / 17
11 Features of Zhang & Ping x = [ x 1 x 2 x 3 ] [Zhang,Ping-2003] P (y x, β) Binomial distribution Gaussian approximation Embedding invariants: x 1 + x 2, x 3 P x (x) precover assumption [Ker-2007] 7 / 17
12 Features of Zhang & Ping Emb(β) x = [ x 1 x 2 x 3 ] β 1 β y = [ x β 1 x β 2 x β 3 ] [Zhang,Ping-2003] arg max β 0 P (y x, β)p x(x)dx P (y x, β) Binomial distribution Gaussian approximation Embedding invariants: x 1 + x 2, x 3 P x (x) precover assumption [Ker-2007] Precover 1/2 1/2 x 1 x 2 + x 3 7 / 17
13 Performance Evaluation Median absolute error 10 2 Jsteg ML - Zhang & Ping Change rate β 3,250 JPEG images resized and compressed to QF=75 Performance similar to [Zhang,Ping-2003] Assumption x β 1 = expected value Zhang & Ping s estimator 8 / 17
14 Performance Evaluation Median absolute error 10 2 Jsteg ML - Zhang & Ping SVR Cartesian-calibrated Pevný features (548) Additional 3,250 images for training Change rate β 3,250 JPEG images resized and compressed to QF=75 Performance similar to [Zhang,Ping-2003] Assumption x β 1 = expected value Zhang & Ping s estimator 8 / 17
15 First-Order Statistics x = [ x 2L, x 2L+1,..., x 2R, x 2R+1 ] ˆβ = arg max β 0 P (y x, β)p x(x)dx Embedding changes in individual LSB pairs are independent ( ) ( ) P (y x, β) = P x β 0 x 0, β P x β 1 x 1, β P Embedding invariants: x 0, x 1, x 2k + x 2k+1 Binomial distribution Gaussian approximation k ( ) x β 2k, xβ 2k+1 x 2k, x 2k+1, β 9 / 17
16 First-Order Statistics x = [ x 2L, x 2L+1,..., x 2R, x 2R+1 ] ˆβ = arg max β 0 P (y x, β)p x(x)dx p 1 2s ( x + 1 ) p s DCT coefficients are i.i.d. drawn from generalized Cauchy distribution Parameters p and s are ML estimates, given embedding invariants Precover assumption for every LSB pair Precover Embedding invariants: x 0, x 1, x 2k + x 2k+1 9 / 17
17 Performance Evaluation Median absolute error 10 2 Jsteg ML - Zhang & Ping ML - First-order SVR Change rate β 10 / 17
18 Second-Order Statistics DCT coefficients are not i.i.d. We capture dependencies using adjacency matrix X Natural decomposition into k-nodes, k {1, 2, 4} Binomial / multinomial distributions Gaussian approximations [-2,3] [-1,3] [0,3] [1,3] [2,3] [3,3] [-2,2] [-1,2] [0,2] [1,2] [2,2] [3,2] arg max β 0 P (Y X, β)p x(x)dx [-2,1] [-1,1] [0,1] [1,1] [2,1] [3,1] [-2,0] [-1,0] [0,0] [1,0] [2,0] [3,0] Factorization of P (y x, β) [-2,-1] [-1,-1] [0,-1] [1,-1] [2,-1] [3,-1] Embedding invariants [-2,-2] [-1,-2] [0,-2] [1,-2] [2,-2] [3,-2] Analytic expression 11 / 17
19 Second-Order Statistics DCT coefficients are not i.i.d. We capture dependencies using adjacency matrix X Natural decomposition into k-nodes, k {1, 2, 4} Binomial / multinomial distributions Gaussian approximations [-2,3] [-1,3] [0,3] [1,3] [2,3] [3,3] [-2,2] [-1,2] [0,2] [1,2] [2,2] [3,2] arg max β 0 P (Y X, β)p x(x)dx [-2,1] [-1,1] [0,1] [1,1] [2,1] [3,1] [-2,0] [-1,0] [0,0] [1,0] [2,0] [3,0] Complications arise [-2,-1] [-1,-1] [0,-1] [1,-1] [2,-1] [3,-1] Good parametric model? [-2,-2] [-1,-2] [0,-2] [1,-2] [2,-2] [3,-2] High complexity 11 / 17
20 Zero Message Hypothesis (ZMH) Alternative heuristic approach Penalty function z(x) 0 satisfying z(x β ) 0 when β = 0 z(x β ) > 0 when β > 0 z(x) should be a quantitative description of a zero message hypothesis capturing a key cover property violated by embedding Assumption: y = E[x β ] = Emb(x, β) Assumption: mapping Emb is invertible x = Emb 1 (y, β) Comments ˆβ = arg min β 0 z(emb 1 (y, β)) Low computational complexity one-dimensional search over β ZMH-based steganalysis is not a new idea! [RS steganalysis,2001] 12 / 17
21 First-Order Statistics (ZMH) x = [x 2L, x 2L+1,..., x 2R 1, x 2R ] Penalty function z sym (x) = w k (x k x k ) 2 Weights w k chosen to minimize the estimator variance least squares steganalysis [Ker-2007] Final form of the penalty function: z sym (x) = k>0 (x k x k ) 2 x k + x k 13 / 17
22 Performance Evaluation Median absolute error 10 2 Jsteg ML - Zhang & Ping ML - First-order ZMH - First-order SVR Change rate β 14 / 17
23 Second-Order Statistics (ZMH) Feature vector: adjacency matrix X ZMH approach Decomposition into k-nodes Embedding is invertible provided 0 β < 1/2 Symmetry about D [-2,3] [-2,2] [-1,3] [-1,2] [0,3] [0,2] [1,3] [1,2] [2,3] [2,2] [3,3] [3,2] [-2,1] [-1,1] [0,1] [1,1] [2,1] [3,1] z adj (X) = i,j (x i,j x j, i ) 2 x i,j + x j, i [-2,0] [-1,0] [0,0] [1,0] [2,0] [3,0] [-2,-1] [-1,-1] [0,-1] [1,-1] [2,-1] [3,-1] [-2,-2] [-1,-2] [0,-2] [1,-2] [2,-2] [3,-2] D 15 / 17
24 Second-Order Statistics (ZMH) Feature vector: adjacency matrix X ZMH approach Decomposition into k-nodes Embedding is invertible provided 0 β < 1/2 Symmetry about D [-2,3] [-2,2] [-1,3] [-1,2] [0,3] [0,2] [1,3] [1,2] [2,3] [2,2] [3,3] [3,2] [-2,1] [-1,1] [0,1] [1,1] [2,1] [3,1] z adj (X) = i,j (x i,j x j, i ) 2 x i,j + x j, i [-2,0] [-1,0] [0,0] [1,0] [2,0] [3,0] [-2,-1] [-1,-1] [0,-1] [1,-1] [2,-1] [3,-1] [-2,-2] [-1,-2] [0,-2] [1,-2] [2,-2] [3,-2] D 15 / 17
25 Performance Evaluation Median absolute error 10 2 Jsteg ML - Zhang & Ping ML - First-order ZMH - First-order ZMH - Second-order SVR Change rate β 16 / 17
26 What Else Can You Find in the Paper / Journal Version Error analysis of between-image and within-image errors for selected attacks Verification of precover assumptions using two different statistical tests Discussion & experiments with the symmetrized version of Jsteg Conversion of the Category attack [Lee-2006] into a quantitative one through the proposed ZMH framework Experiments conducted on two different sources of images Results reported in terms of two more security measures: IQR, median bias 17 / 17
Quantitative Structural Steganalysis of Jsteg
Quantitative Structural Steganalysis of Jsteg Jan Kodovský and Jessica Fridrich, Member, IEEE Abstract Quantitative steganalysis strives to estimate the change rate defined as the relative number of embedding
More informationFisher Information Determines Capacity of ε-secure Steganography
Fisher Information Determines Capacity of ε-secure Steganography Tomáš Filler and Jessica Fridrich Dept. of Electrical and Computer Engineering SUNY Binghamton, New York 11th Information Hiding, Darmstadt,
More informationComplete characterization of perfectly secure stego-systems with mutually independent embedding operation
Complete characterization of perfectly secure stego-systems with mutually independent embedding operation Tomáš Filler and Jessica Fridrich Dept. of Electrical and Computer Engineering SUNY Binghamton,
More informationJPEG-Compatibility Steganalysis Using Block-Histogram of Recompression Artifacts
JPEG-Compatibility Steganalysis Using Block-Histogram of Recompression Artifacts Jan Kodovský and Jessica Fridrich Department of ECE, Binghamton University, NY, USA {fridrich,jan.kodovsky}@binghamton.edu
More informationGeneralized Transfer Component Analysis for Mismatched Jpeg Steganalysis
Generalized Transfer Component Analysis for Mismatched Jpeg Steganalysis Xiaofeng Li 1, Xiangwei Kong 1, Bo Wang 1, Yanqing Guo 1,Xingang You 2 1 School of Information and Communication Engineering Dalian
More informationROC Curves for Steganalysts
ROC Curves for Steganalysts Andreas Westfeld Technische Universität Dresden Institut für Systemarchitektur 01062 Dresden mailto:westfeld@inf.tu-dresden.de 1 Introduction There are different approaches
More informationImage Data Compression. Steganography and steganalysis Alexey Pak, PhD, Lehrstuhl für Interak;ve Echtzeitsysteme, Fakultät für Informa;k, KIT
Image Data Compression Steganography and steganalysis 1 Stenography vs watermarking Watermarking: impercep'bly altering a Work to embed a message about that Work Steganography: undetectably altering a
More informationThe Square Root Law of Steganographic Capacity
The Square Root Law of Steganographic Capacity ABSTRACT Andrew D. Ker Oxford University Computing Laboratory Parks Road Oxford OX1 3QD, UK adk@comlab.ox.ac.uk Jan Kodovský Dept. of Electrical and Computer
More informationGaussian Processes. Le Song. Machine Learning II: Advanced Topics CSE 8803ML, Spring 2012
Gaussian Processes Le Song Machine Learning II: Advanced Topics CSE 8803ML, Spring 01 Pictorial view of embedding distribution Transform the entire distribution to expected features Feature space Feature
More informationInformation Hiding and Covert Communication
Information Hiding and Covert Communication Andrew Ker adk @ comlab.ox.ac.uk Royal Society University Research Fellow Oxford University Computing Laboratory Foundations of Security Analysis and Design
More informationOptimal Detection of OutGuess using an Accurate Model of DCT Coefficients
Optimal Detection of OutGuess using an Accurate Model of DCT Coefficients Thanh Hai Thai, Rémi Cogranne, Florent Retraint ICD - ROSAS - LMS - Troyes University of Technology - UMR 68, CNRS Troyes - France
More informationTheoretical Model of the FLD Ensemble Classifier Based on Hypothesis Testing Theory
Theoretical Model of the FLD Ensemble Classifier Based on Hypothesis Testing Theory Rémi Cogranne, Member, IEEE, ICD - ROSAS - LM2S Troyes University of Technology - UMR 6281, CNRS 12, rue Marie Curie
More informationDetection of Content Adaptive LSB Matching (a Game Theory Approach)
Detection of Content Adaptive LSB Matching (a Game Theory Approach) Tomáš Denemark and Jessica Fridrich Department of ECE, SUNY Binghamton, NY, USA ABSTRACT This paper is an attempt to analyze the interaction
More informationMachine Learning Basics: Maximum Likelihood Estimation
Machine Learning Basics: Maximum Likelihood Estimation Sargur N. srihari@cedar.buffalo.edu This is part of lecture slides on Deep Learning: http://www.cedar.buffalo.edu/~srihari/cse676 1 Topics 1. Learning
More informationMachine Learning. Lecture 4: Regularization and Bayesian Statistics. Feng Li. https://funglee.github.io
Machine Learning Lecture 4: Regularization and Bayesian Statistics Feng Li fli@sdu.edu.cn https://funglee.github.io School of Computer Science and Technology Shandong University Fall 207 Overfitting Problem
More informationDetection of Content Adaptive LSB Matching (a Game Theory Approach)
Detection of Content Adaptive LSB Matching (a Game Theory Approach) Tomáš Denemark and Jessica Fridrich Department of ECE, SUNY Binghamton, NY, USA ABSTRACT This paper is an attempt to analyze the interaction
More informationLearning outcomes. Palettes and GIF. The colour palette. Using the colour palette The GIF file. CSM25 Secure Information Hiding
Learning outcomes Palettes and GIF CSM25 Secure Information Hiding Dr Hans Georg Schaathun University of Surrey Learn how images are represented using a palette Get an overview of hiding techniques in
More informationImproved Adaptive LSB Steganography based on Chaos and Genetic Algorithm
Improved Adaptive LSB Steganography based on Chaos and Genetic Algorithm Lifang Yu, Yao Zhao 1, Rongrong Ni, Ting Li Institute of Information Science, Beijing Jiaotong University, BJ 100044, China Abstract
More informationLecture 3: Statistical Decision Theory (Part II)
Lecture 3: Statistical Decision Theory (Part II) Hao Helen Zhang Hao Helen Zhang Lecture 3: Statistical Decision Theory (Part II) 1 / 27 Outline of This Note Part I: Statistics Decision Theory (Classical
More information2. Definition & Classification
Information Hiding Data Hiding K-H Jung Agenda 1. Data Hiding 2. Definition & Classification 3. Related Works 4. Considerations Definition of Data Hiding 3 Data Hiding, Information Hiding Concealing secret
More informationSteganalysis of Spread Spectrum Data Hiding Exploiting Cover Memory
Steganalysis of Spread Spectrum Data Hiding Exploiting Cover Memory Kenneth Sullivan, Upamanyu Madhow, Shivkumar Chandrasekaran, and B.S. Manjunath Department of Electrical and Computer Engineering University
More informationSteganalysis with a Computational Immune System
DIGITAL FORENSIC RESEARCH CONFERENCE Steganalysis with a Computational Immune System By Jacob Jackson, Gregg Gunsch, Roger Claypoole, Gary Lamont Presented At The Digital Forensic Research Conference DFRWS
More informationMINIMIZING ADDITIVE DISTORTION FUNCTIONS WITH NON-BINARY EMBEDDING OPERATION IN STEGANOGRAPHY. Tomáš Filler and Jessica Fridrich
MINIMIZING ADDITIVE DISTORTION FUNCTIONS WITH NON-BINARY EMBEDDING OPERATION IN STEGANOGRAPHY Tomáš Filler and Jessica Fridrich Department of ECE, SUNY Binghamton, NY, USA {tomas.filler, fridrich}@binghamton.edu
More informationImproving Selection-Channel-Aware Steganalysis Features
Improving Selection-Channel-Aware Steganalysis Features Tomáš Denemark and Jessica Fridrich, Department of ECE, SUNY Binghamton, NY, USA, {tdenema1,fridrich}@binghamton.edu, Pedro Comesaña-Alfaro, Department
More informationPATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 2: PROBABILITY DISTRIBUTIONS
PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 2: PROBABILITY DISTRIBUTIONS Parametric Distributions Basic building blocks: Need to determine given Representation: or? Recall Curve Fitting Binary Variables
More information6.867 Machine Learning
6.867 Machine Learning Problem set 1 Solutions Thursday, September 19 What and how to turn in? Turn in short written answers to the questions explicitly stated, and when requested to explain or prove.
More informationA Bayesian Perspective on Residential Demand Response Using Smart Meter Data
A Bayesian Perspective on Residential Demand Response Using Smart Meter Data Datong-Paul Zhou, Maximilian Balandat, and Claire Tomlin University of California, Berkeley [datong.zhou, balandat, tomlin]@eecs.berkeley.edu
More informationImage Dependent Log-likelihood Ratio Allocation for Repeat Accumulate Code based Decoding in Data Hiding Channels
Image Dependent Log-likelihood Ratio Allocation for Repeat Accumulate Code based Decoding in Data Hiding Channels Anindya Sarkar and B. S. Manjunath Department of Electrical and Computer Engineering, University
More informationImperfect Stegosystems
DISSERTATION DEFENSE Imperfect Stegosystems Asymptotic Laws and Near-Optimal Practical Constructions Tomá² Filler Dept. of Electrical and Computer Engineering SUNY Binghamton, New York April 1, 2011 2
More informationUSING MULTIPLE RE-EMBEDDINGS FOR QUANTITATIVE STEGANALYSIS AND IMAGE RELIABILITY ESTIMATION
TKK Reports in Information and Computer Science Espoo 2010 TKK-ICS-R34 USING MULTIPLE RE-EMBEDDINGS FOR QUANTITATIVE STEGANALYSIS AND IMAGE RELIABILITY ESTIMATION Yoan Miche, Patrick Bas and Amaury Lendasse
More information6.867 Machine Learning
6.867 Machine Learning Problem set 1 Due Thursday, September 19, in class What and how to turn in? Turn in short written answers to the questions explicitly stated, and when requested to explain or prove.
More informationHypothesis testing:power, test statistic CMS:
Hypothesis testing:power, test statistic The more sensitive the test, the better it can discriminate between the null and the alternative hypothesis, quantitatively, maximal power In order to achieve this
More informationIEOR 165 Lecture 7 1 Bias-Variance Tradeoff
IEOR 165 Lecture 7 Bias-Variance Tradeoff 1 Bias-Variance Tradeoff Consider the case of parametric regression with β R, and suppose we would like to analyze the error of the estimate ˆβ in comparison to
More informationDetection of Content Adaptive LSB Matching (a Game Theory Approach)
Detection of Content Adaptive LSB Matching (a Game Theory Approach) Tomáš Denemark, Jessica Fridrich / 5 Content-adaptive steganography Every pixel is changed with probability β i = exp( λρ i) + exp( λρ
More informationLow Complexity Features for JPEG Steganalysis Using Undecimated DCT
1 Low Complexity Features for JPEG Steganalysis Using Undecimated DCT Vojtěch Holub and Jessica Fridrich, Member, IEEE Abstract This article introduces a novel feature set for steganalysis of JPEG images
More informationResearch Article Improved Adaptive LSB Steganography Based on Chaos and Genetic Algorithm
Hindawi Publishing Corporation EURASIP Journal on Advances in Signal Processing Volume 2010, Article ID 876946, 6 pages doi:10.1155/2010/876946 Research Article Improved Adaptive LSB Steganography Based
More informationDIGITAL image processing has remarkably developed
IEEE TRANSACTION ON IMAGE PROCESSING SUBMITTED VERSION, OCT. 3 Statistical Model of Quantized DCT Coefficients : Application in the Steganalysis of Jsteg Algorithm Thanh Hai Thai, Rémi Cogranne, Florent
More informationMachine Learning. Linear Models. Fabio Vandin October 10, 2017
Machine Learning Linear Models Fabio Vandin October 10, 2017 1 Linear Predictors and Affine Functions Consider X = R d Affine functions: L d = {h w,b : w R d, b R} where ( d ) h w,b (x) = w, x + b = w
More informationAccouncements. You should turn in a PDF and a python file(s) Figure for problem 9 should be in the PDF
Accouncements You should turn in a PDF and a python file(s) Figure for problem 9 should be in the PDF Please do not zip these files and submit (unless there are >5 files) 1 Bayesian Methods Machine Learning
More informationCONTENT ADAPTIVE STEGANOGRAPHY DESIGN AND DETECTION
CONTENT ADAPTIVE STEGANOGRAPHY DESIGN AND DETECTION BY VOJTĚCH HOLUB MS, Czech Technical University, Prague, 2010 DISSERTATION Submitted in partial fulfillment of the requirements for the degree of Doctor
More informationObtaining Higher Rates for Steganographic Schemes while Maintaining the Same Detectability
Obtaining Higher Rates for Steganographic Schemes while Maintaining the Same Detectability Anindya Sarkar, Kaushal Solanki and and B. S. Manjunath Department of Electrical and Computer Engineering, University
More informationIntroduction to Statistical Learning Theory
Introduction to Statistical Learning Theory In the last unit we looked at regularization - adding a w 2 penalty. We add a bias - we prefer classifiers with low norm. How to incorporate more complicated
More informationLecture 5. Ideal and Almost-Ideal SG.
Lecture 5. Ideal and Almost-Ideal SG. Definition. SG is called ideal (perfect or unconditionally undetectable ISG),if its detection is equivalently to random guessing of this fact even with the use of
More informationMatrix Embedding with Pseudorandom Coefficient Selection and Error Correction for Robust and Secure Steganography
Matrix Embedding with Pseudorandom Coefficient Selection and Error Correction for Robust and Secure Steganography Anindya Sarkar, Student Member, IEEE, Upamanyu Madhow, Fellow, IEEE, and B. S. Manjunath,
More informationLecture 4: Types of errors. Bayesian regression models. Logistic regression
Lecture 4: Types of errors. Bayesian regression models. Logistic regression A Bayesian interpretation of regularization Bayesian vs maximum likelihood fitting more generally COMP-652 and ECSE-68, Lecture
More informationSteganalysis of ±k Steganography based on Noncausal Linear Predictor
INTERNATIONAL JOURNAL OF COMPUTERS COMMUNICATIONS & CONTROL ISSN 1841-9836, 9(5):623-632, October, 2014. Steganalysis of ±k Steganography based on Noncausal Linear Predictor K.M. Singh, Y.J. Chanu, T.
More informationSparse Linear Models (10/7/13)
STA56: Probabilistic machine learning Sparse Linear Models (0/7/) Lecturer: Barbara Engelhardt Scribes: Jiaji Huang, Xin Jiang, Albert Oh Sparsity Sparsity has been a hot topic in statistics and machine
More informationLecture : Probabilistic Machine Learning
Lecture : Probabilistic Machine Learning Riashat Islam Reasoning and Learning Lab McGill University September 11, 2018 ML : Many Methods with Many Links Modelling Views of Machine Learning Machine Learning
More informationParametric Models. Dr. Shuang LIANG. School of Software Engineering TongJi University Fall, 2012
Parametric Models Dr. Shuang LIANG School of Software Engineering TongJi University Fall, 2012 Today s Topics Maximum Likelihood Estimation Bayesian Density Estimation Today s Topics Maximum Likelihood
More informationNaïve Bayes classification
Naïve Bayes classification 1 Probability theory Random variable: a variable whose possible values are numerical outcomes of a random phenomenon. Examples: A person s height, the outcome of a coin toss
More informationCOMS 4771 Regression. Nakul Verma
COMS 4771 Regression Nakul Verma Last time Support Vector Machines Maximum Margin formulation Constrained Optimization Lagrange Duality Theory Convex Optimization SVM dual and Interpretation How get the
More informationA Reservoir Sampling Algorithm with Adaptive Estimation of Conditional Expectation
A Reservoir Sampling Algorithm with Adaptive Estimation of Conditional Expectation Vu Malbasa and Slobodan Vucetic Abstract Resource-constrained data mining introduces many constraints when learning from
More informationPerturbed Quantization Steganography
Perturbed Quantization Steganography Jessica Fridrich SUNY Binghamton Department of ECE Binghamton, NY 13902-6000 001 607 777 2577 fridrich@binghamton.edu Miroslav Goljan SUNY Binghamton Department of
More informationLinear Models for Regression CS534
Linear Models for Regression CS534 Prediction Problems Predict housing price based on House size, lot size, Location, # of rooms Predict stock price based on Price history of the past month Predict the
More informationNew Steganographic scheme based of Reed- Solomon codes
New Steganographic scheme based of Reed- Solomon codes I. DIOP; S.M FARSSI ;O. KHOUMA ; H. B DIOUF ; K.TALL ; K.SYLLA Ecole Supérieure Polytechnique de l Université Dakar Sénégal Email: idydiop@yahoo.fr;
More informationRegression I: Mean Squared Error and Measuring Quality of Fit
Regression I: Mean Squared Error and Measuring Quality of Fit -Applied Multivariate Analysis- Lecturer: Darren Homrighausen, PhD 1 The Setup Suppose there is a scientific problem we are interested in solving
More informationA Study of Embedding Operations and Locations for Steganography in H.264 Video
A Study of Embedding Operations and Locations for Steganography in H.264 Video Andreas Neufeld and Andrew D. Ker Oxford University Department of Computer Science Andreas Neufeld is now with the Image and
More informationSTATISTICAL DETECTION OF INFORMATION HIDING BASED ON ADJACENT PIXELS DIFFERENCE
2th European Signal Processing Conference EUSIPCO 22) Bucharest, Romania, August 27-3, 22 STATISTICAL DETECTION OF INFORMATION HIDING BASED ON ADJACENT PIXELS DIFFERENCE Rémi Cogranne, Cathel Zitzmann,
More informationLecture 3: More on regularization. Bayesian vs maximum likelihood learning
Lecture 3: More on regularization. Bayesian vs maximum likelihood learning L2 and L1 regularization for linear estimators A Bayesian interpretation of regularization Bayesian vs maximum likelihood fitting
More informationIntroduction to Machine Learning. Lecture 2
Introduction to Machine Learning Lecturer: Eran Halperin Lecture 2 Fall Semester Scribe: Yishay Mansour Some of the material was not presented in class (and is marked with a side line) and is given for
More informationDetecting LSB Matching by Applying Calibration Technique for Difference Image
Detecting LSB atching by Applying Calibration Technique for Difference Image iaolong Li Institute of Computer Science & Technology Peking University 87, Beijing, P. R. China lixiaolong@icst.pku.edu.cn
More informationData Mining Stat 588
Data Mining Stat 588 Lecture 02: Linear Methods for Regression Department of Statistics & Biostatistics Rutgers University September 13 2011 Regression Problem Quantitative generic output variable Y. Generic
More informationReview. DS GA 1002 Statistical and Mathematical Models. Carlos Fernandez-Granda
Review DS GA 1002 Statistical and Mathematical Models http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall16 Carlos Fernandez-Granda Probability and statistics Probability: Framework for dealing with
More informationEXAM IN STATISTICAL MACHINE LEARNING STATISTISK MASKININLÄRNING
EXAM IN STATISTICAL MACHINE LEARNING STATISTISK MASKININLÄRNING DATE AND TIME: August 30, 2018, 14.00 19.00 RESPONSIBLE TEACHER: Niklas Wahlström NUMBER OF PROBLEMS: 5 AIDING MATERIAL: Calculator, mathematical
More informationHigh-dimensional regression
High-dimensional regression Advanced Methods for Data Analysis 36-402/36-608) Spring 2014 1 Back to linear regression 1.1 Shortcomings Suppose that we are given outcome measurements y 1,... y n R, and
More informationProbabilistic modeling. The slides are closely adapted from Subhransu Maji s slides
Probabilistic modeling The slides are closely adapted from Subhransu Maji s slides Overview So far the models and algorithms you have learned about are relatively disconnected Probabilistic modeling framework
More informationNaïve Bayes classification. p ij 11/15/16. Probability theory. Probability theory. Probability theory. X P (X = x i )=1 i. Marginal Probability
Probability theory Naïve Bayes classification Random variable: a variable whose possible values are numerical outcomes of a random phenomenon. s: A person s height, the outcome of a coin toss Distinguish
More informationSupport Vector Machine I
Support Vector Machine I Jia-Bin Huang ECE-5424G / CS-5824 Virginia Tech Spring 2019 Administrative Please use piazza. No emails. HW 0 grades are back. Re-grade request for one week. HW 1 due soon. HW
More informationSTATISTICS SYLLABUS UNIT I
STATISTICS SYLLABUS UNIT I (Probability Theory) Definition Classical and axiomatic approaches.laws of total and compound probability, conditional probability, Bayes Theorem. Random variable and its distribution
More informationMachine Learning CSE546 Carlos Guestrin University of Washington. September 30, 2013
Bayesian Methods Machine Learning CSE546 Carlos Guestrin University of Washington September 30, 2013 1 What about prior n Billionaire says: Wait, I know that the thumbtack is close to 50-50. What can you
More informationIntroduction to Statistical modeling: handout for Math 489/583
Introduction to Statistical modeling: handout for Math 489/583 Statistical modeling occurs when we are trying to model some data using statistical tools. From the start, we recognize that no model is perfect
More informationModern Methods of Data Analysis - WS 07/08
Modern Methods of Data Analysis Lecture VII (26.11.07) Contents: Maximum Likelihood (II) Exercise: Quality of Estimators Assume hight of students is Gaussian distributed. You measure the size of N students.
More informationSTA414/2104 Statistical Methods for Machine Learning II
STA414/2104 Statistical Methods for Machine Learning II Murat A. Erdogdu & David Duvenaud Department of Computer Science Department of Statistical Sciences Lecture 3 Slide credits: Russ Salakhutdinov Announcements
More informationStatistical learning. Chapter 20, Sections 1 3 1
Statistical learning Chapter 20, Sections 1 3 Chapter 20, Sections 1 3 1 Outline Bayesian learning Maximum a posteriori and maximum likelihood learning Bayes net learning ML parameter learning with complete
More information3 Joint Distributions 71
2.2.3 The Normal Distribution 54 2.2.4 The Beta Density 58 2.3 Functions of a Random Variable 58 2.4 Concluding Remarks 64 2.5 Problems 64 3 Joint Distributions 71 3.1 Introduction 71 3.2 Discrete Random
More informationReminders. Thought questions should be submitted on eclass. Please list the section related to the thought question
Linear regression Reminders Thought questions should be submitted on eclass Please list the section related to the thought question If it is a more general, open-ended question not exactly related to a
More informationLecture 7 Introduction to Statistical Decision Theory
Lecture 7 Introduction to Statistical Decision Theory I-Hsiang Wang Department of Electrical Engineering National Taiwan University ihwang@ntu.edu.tw December 20, 2016 1 / 55 I-Hsiang Wang IT Lecture 7
More informationInformation Hiding. Andrew D. Ker. 10 Lectures, Hilary Term (complete) Department of Computer Science, Oxford University
Information Hiding (complete) Andrew D. Ker 10 Lectures, Hilary Term 2016 Department of Computer Science, Oxford University ii Contents 0 Introduction 1 0.1 Preliminary Material..............................
More informationGaussians Linear Regression Bias-Variance Tradeoff
Readings listed in class website Gaussians Linear Regression Bias-Variance Tradeoff Machine Learning 10701/15781 Carlos Guestrin Carnegie Mellon University January 22 nd, 2007 Maximum Likelihood Estimation
More informationCSC2515 Winter 2015 Introduction to Machine Learning. Lecture 2: Linear regression
CSC2515 Winter 2015 Introduction to Machine Learning Lecture 2: Linear regression All lecture slides will be available as.pdf on the course website: http://www.cs.toronto.edu/~urtasun/courses/csc2515/csc2515_winter15.html
More informationDS-GA 1002 Lecture notes 11 Fall Bayesian statistics
DS-GA 100 Lecture notes 11 Fall 016 Bayesian statistics In the frequentist paradigm we model the data as realizations from a distribution that depends on deterministic parameters. In contrast, in Bayesian
More informationConfidence Intervals for Low-dimensional Parameters with High-dimensional Data
Confidence Intervals for Low-dimensional Parameters with High-dimensional Data Cun-Hui Zhang and Stephanie S. Zhang Rutgers University and Columbia University September 14, 2012 Outline Introduction Methodology
More informationPattern Recognition. Parameter Estimation of Probability Density Functions
Pattern Recognition Parameter Estimation of Probability Density Functions Classification Problem (Review) The classification problem is to assign an arbitrary feature vector x F to one of c classes. The
More informationMachine Learning. Gaussian Mixture Models. Zhiyao Duan & Bryan Pardo, Machine Learning: EECS 349 Fall
Machine Learning Gaussian Mixture Models Zhiyao Duan & Bryan Pardo, Machine Learning: EECS 349 Fall 2012 1 The Generative Model POV We think of the data as being generated from some process. We assume
More informationLinear Regression. CSL603 - Fall 2017 Narayanan C Krishnan
Linear Regression CSL603 - Fall 2017 Narayanan C Krishnan ckn@iitrpr.ac.in Outline Univariate regression Multivariate regression Probabilistic view of regression Loss functions Bias-Variance analysis Regularization
More informationLinear Regression. CSL465/603 - Fall 2016 Narayanan C Krishnan
Linear Regression CSL465/603 - Fall 2016 Narayanan C Krishnan ckn@iitrpr.ac.in Outline Univariate regression Multivariate regression Probabilistic view of regression Loss functions Bias-Variance analysis
More informationMotivation Scale Mixutres of Normals Finite Gaussian Mixtures Skew-Normal Models. Mixture Models. Econ 690. Purdue University
Econ 690 Purdue University In virtually all of the previous lectures, our models have made use of normality assumptions. From a computational point of view, the reason for this assumption is clear: combined
More informationSparse regression. Optimization-Based Data Analysis. Carlos Fernandez-Granda
Sparse regression Optimization-Based Data Analysis http://www.cims.nyu.edu/~cfgranda/pages/obda_spring16 Carlos Fernandez-Granda 3/28/2016 Regression Least-squares regression Example: Global warming Logistic
More informationApproximate Kernel PCA with Random Features
Approximate Kernel PCA with Random Features (Computational vs. Statistical Tradeoff) Bharath K. Sriperumbudur Department of Statistics, Pennsylvania State University Journées de Statistique Paris May 28,
More informationMinimizing Embedding Impact in Steganography using Trellis-Coded Quantization
Minimizing Embedding Impact in Steganography using Trellis-Coded Quantization Tomáš Filler, Jan Judas, and Jessica Fridrich Department of Electrical and Computer Engineering SUNY Binghamton, Binghamton,
More informationLecture 5: Logistic Regression. Neural Networks
Lecture 5: Logistic Regression. Neural Networks Logistic regression Comparison with generative models Feed-forward neural networks Backpropagation Tricks for training neural networks COMP-652, Lecture
More informationDouble Gamma Principal Components Analysis
Applied Mathematical Sciences, Vol. 12, 2018, no. 11, 523-533 HIKARI Ltd, www.m-hikari.com https://doi.org/10.12988/ams.2018.8455 Double Gamma Principal Components Analysis Ameerah O. Bahashwan, Zakiah
More informationStatistical Machine Learning from Data
Samy Bengio Statistical Machine Learning from Data 1 Statistical Machine Learning from Data Ensembles Samy Bengio IDIAP Research Institute, Martigny, Switzerland, and Ecole Polytechnique Fédérale de Lausanne
More informationSTA 414/2104: Machine Learning
STA 414/2104: Machine Learning Russ Salakhutdinov Department of Computer Science! Department of Statistics! rsalakhu@cs.toronto.edu! http://www.cs.toronto.edu/~rsalakhu/ Lecture 9 Sequential Data So far
More informationBayesian Machine Learning
Bayesian Machine Learning Andrew Gordon Wilson ORIE 6741 Lecture 4 Occam s Razor, Model Construction, and Directed Graphical Models https://people.orie.cornell.edu/andrew/orie6741 Cornell University September
More informationLecture 2 Machine Learning Review
Lecture 2 Machine Learning Review CMSC 35246: Deep Learning Shubhendu Trivedi & Risi Kondor University of Chicago March 29, 2017 Things we will look at today Formal Setup for Supervised Learning Things
More informationSteganalysis in JPEG
Steganalysis in JPEG CSM25 Secure Information Hiding Dr Hans Georg Schaathun University of Surrey Spring 2007 Dr Hans Georg Schaathun Steganalysis in JPEG Spring 2007 1 / 24 Learning Outcomes learn additional
More informationBayesian Approach 2. CSC412 Probabilistic Learning & Reasoning
CSC412 Probabilistic Learning & Reasoning Lecture 12: Bayesian Parameter Estimation February 27, 2006 Sam Roweis Bayesian Approach 2 The Bayesian programme (after Rev. Thomas Bayes) treats all unnown quantities
More informationCompressed Sensing and Linear Codes over Real Numbers
Compressed Sensing and Linear Codes over Real Numbers Henry D. Pfister (joint with Fan Zhang) Texas A&M University College Station Information Theory and Applications Workshop UC San Diego January 31st,
More informationMultiple regression. CM226: Machine Learning for Bioinformatics. Fall Sriram Sankararaman Acknowledgments: Fei Sha, Ameet Talwalkar
Multiple regression CM226: Machine Learning for Bioinformatics. Fall 2016 Sriram Sankararaman Acknowledgments: Fei Sha, Ameet Talwalkar Multiple regression 1 / 36 Previous two lectures Linear and logistic
More information