1 Hoeffding s Inequality
|
|
- Dinah Todd
- 5 years ago
- Views:
Transcription
1 Proailistic Method: Hoeffding s Inequality and Differential Privacy Lecturer: Huert Chan Date: 27 May 22 Hoeffding s Inequality. Approximate Counting y Random Sampling Suppose there is a ag containing red alls and lue alls. You would like to estimate the fraction of red alls in the ag. However, you are only allowed to sample randomly from the ag with replacement. The straightforward method is to make T random samples and use the fraction of red alls in the random samples as an estimate of the true fraction. We are interested in estimation with certain additive error, i.e., if the true fraction is p, it is enough to return a numer in the range [p, p + ]. The question is: how many samples are enough so that with proaility at least δ, our estimate is within an additive error of > from the true value? Let Y := t [T ] Z t, where Z t is the {, }-random variale that takes value if the t-th sample is red. The estimator is Y T. Suppose the true fraction of red alls in the ag is p. Then, we want to find how large T needs to e such that the following is satisfied: P r[ Y T p ] δ. Theorem. Hoeffding s Inequality Suppose X, X 2,..., X n are independent real-valued random variales, such that for each i, X i takes values from the interval [a i, i ]. Let Y := i X i. Then, for all α >, P r[ Y E[Y ] nα] 2 exp 2n2 α 2, i R2 i where R i := i a i. Using the Hoeffding s Inequality, with X i [, ] and E[X i ] = p, we have P r[ T i X i p α] 2 exp 2T α 2. Hence, in order to estimate the fraction of red alls with additive error at most α and failure proaility at most δ, it suffices to use T = Θ log α 2 δ..2 Proof of Hoeffding s Inequality We use the technique of moment generating function to prove the Hoeffding s Inequality. simplicity, we prove a slightly weaker result: P r[ Y E[Y ] nα] 2 exp n2 α 2 2. i R2 i Here are the three steps. For
2 .2. Transform the Inequality into a Convenient Form We use the inequality P r[ Y E[Y ] nα] P r[y E[Y ] nα] + P r[y E[Y ] nα], and give ounds for the two proailities separately. Here, we consider P r[y E[Y ] nα]. The other case is similar. Oserve that the expectation of Y does not appear in the upperound. It would e more convenient to first translate the variale X i. Define Z i := X i E[X i ]. Oserve that E[Z i ] =. Moreover, since oth X i and E[X i ] are in the interval [a i, i ], it follows that Z i takes values in the interval [ R i, R i ]. For t >, we have P r[y E[Y ] nα] = P r[t i Z i tnα]..2.2 Using Moment Generating Function and Independence Applying the exponentiation function to oth sides of the inequality, we follow the standard calculation, using independence of the Z i s. P r[t i Z i tnα] = P r[expt i Z i exptnα] exp tnα E[expt Z i ] i exp tnα E[exptZ i ] i The next step is the most technical part of the proof. Recall that we want to find some appropriate function g i t such that E[exptZ i ] expg i t. All we know aout Z i is that E[Z i ] = and Z i takes value in [ R i, R i ]. Think of a simple ut non-trivial random variale that has mean and takes values in [ R i, R i ]. Consider Ẑi that takes value R i with proaility 2 and R i with proaility 2. Then, it follows that E[exptẐi] = 2 etr i + e tr i exp 2 t2 Ri 2, the last inequality follows from the fact that for all real x, 2 ex + e x e 2 x2. Therefore, for such a simple Ẑi, we have a nice ound E[exptẐi] expg i t, where g i t := 2 t2 Ri 2. Using Convexity to Show that the Extreme Points are the Worst Case Scenario. Intuitively, Ẑ i is the worst case scenario. Since we wish to show measure concentration, it is a ad case if there is a lot of variation for Z i. However, we have the requirement that E[Z i ] = and Z i [ R i, R i ]. Hence, intuitively, Z i has the most variation if it takes values only at the extreme points, each with proaility 2. We formalize this intuition using the convexity of the exponentiation function. Definition.2 A function f : R R is convex if for all x, y R and λ, 2
3 fλx + λy λfx + λfy. Fact.3 If a function is douly differentiale and f x, then f is convex. We use the fact that the exponentiation function x e tx is convex. First, we need to express Z i as a convex comination of the end points, i.e., we want to find λ such that Z i = λr i + λ R i. We have λ := Z i+r i 2R i. Using the convexity of the function x e tx, we have exptz i = exptλr i + λ R i λ exptr i + λ exp tr i = 2 + Z i 2R i exptr i + 2 Z i 2R i exp tr i. Take expectation on oth sides, and oserving that E[Z i ] =, we have E[exptZ i ] 2 exptr i + exp tr i = E[exptẐi], as required..2.3 Picking the Best Value for t We have shown that for all t >, P r[y E[Y ] nα] exp 2 t2 i R2 i nαt, and we want to find a value of t that minimizes the right hand side. Note that the exponent is a quadratic function in t and hence is minimized when t := Hence, in this case, we have P r[y E[Y ] nα] exp n2 α 2 2, as required. i R2 i.3 Exercise nα i R2 i Estimating the Unknown Fraction of Red Balls. Suppose a ag contains an unknown numer of red alls assume there is at least one red all and you are only allowed to sample with replacement uniformly at random from the ag. Design an algorithm that, given <, δ <, with failure proaility at most δ, returns an estimate of the fraction of red alls with multiplicative error at most, i.e., if the real fraction is p, the algorithm returns a numer p such that p p p. Give the numer of random samples used y your algorithm. 2 Differential Privacy 2. Motivation A statistical dataase is a dataase used for statistical analysis. For example, a dataase containing information aout graduates of a certain university can answer questions like: what is the average salary of the graduates? Statistical dataase are widely used due to the enormous social value they provide: the previously mentioned dataase enefits the society in helping students to choose 3 >.
4 whether to go to that university, or how funding should e distriuted among universities. However, the statistics released might cause leakage of sensitive information. Therefore, a ig challenge is to maintain individual privacy, while providing useful aggregate statistical information aout a certain group. In 977 the statistician Tore Dalenius gave a privacy goal for statistical dataases: anything that can e learned aout a memer in the statistical dataase, should also can e learned without access to the dataase. However, as the following example illustrates, as long as the statistical dataase is useful, the goal is not achievale if the adversary has auxiliary information information not otained from the statistical dataase. Suppose we know that some student s X salary is $, more than the average, we can know his/her salary y querying aout the average salary. Note that in this case, even if X does not join the dataase, we can still approximately know the average salary and hence know his/her sensitive information. Notation. Let U e the set of possile user data points. A dataase of n users contains the data points for each user and can e viewed as a vector in U n. We use D := U n to denote the collection of all possile dataases with n users. We consider whether releasing the output of some function f : D O will compromise an individual s privacy. Example. Suppose there are n users, each of which holds a private it from U = {, }. Given a dataase X U n, suppose the function of interest is sumx = n X i. At first sight, it might seem that releasing the sum does not violate an individual s privacy, ecause the sum does not directly reveal any individual s private it. However, in reality, users data can e in many dataases. Suppose users to n also participate in another dataase which also releases the sum. Then, comining the results of the two sums, the private it of user n can e accurately calculated! Differential privacy defines privacy in a different sense: to minimize the increased risk of the sensitive information leakage due to one s joining in the statistical dataase. A private mechanism that answers queries to statistical dataases can achieve this goal y introducing randomness such that when two dataases differ y only one single user, the output produced have similar distriutions. Note that a differentially private mechanism encourages individuals to participate in statistical dataases, and thus enhancing the social enefit provided y them. 2.2 Formal Definition Let U e the set of possile data points and D := U n e the collection of dataases with n users. Two dataases X D and X 2 D are called neighoring denoted y X X 2, if they differ y at most one coordinate. Definition 2. -differential privacy A randomized mechanism function M : D O preserves -differential privacy, if for any two neighoring dataases X X 2, and any possile set of output S O, the following hold: Pr[MX S] exp Pr[MX 2 S], where the randomness comes from the coin flips of M. Remark 2.2 We oserve the following. 4
5 . Since M is a randomized mechanism, MX is a distriution of outputs in O. 2. Interchanging the roles of X and X 2, we also have: Pr[MX 2 S] exp Pr[MX S]. 3. If O is a countale set, then we can also have the inequality for each x O, Pr[MX = x] exp Pr[MX 2 = x]. 4. The inequality means that the distriutions MX and MX 2 are close, and hence y oserving the output, it is not possile to tell for certain whether the output comes from dataase X or X Achieving Differential Privacy Given a set D of dataases, a deterministic function f : D R d, and a privacy parameter, we want to find an -differentially private version of f, i.e., to find a random function f : D R d that has the following properties:. Privacy. The function f preserves -differential privacy, 2. Utility. For any dataase X D, with high proaility, fx is close to fx. Oserve that the trivial function f satisfies -differential privacy. However, this will not e very useful. Here is one way to define utility. Definition 2.3 λ, δ-useful Let < δ < and λ > e utility parameters. Let f : D R d e a deterministic function and f : D R d e a randomized function. We say f is λ, δ-useful with respect to f, if for any dataase X D, with proaility at least δ, for each i [d], f i X f i X λ. 2.4 Achieving differential privacy y adding Laplace noise One way to convert a function f : D R d into a differentially private version is to add independent random noise to each of its coordinates. Intuitively, the more fx changes when we change one coordinate of X, the larger the random noise is needed to hide the difference. We use l -sensitivity to formally measure the maximum difference etween the values of f of any two neighoring dataases. Definition 2.4 l -Sensitivity Let f : D R d e a deterministic function. The l -sensitivity of f, denoted y f, is max fx fx 2 = max X X 2 X X 2 d f i X f i X 2 We use random variales sampled from Laplace distriutions as the random noise. 5
6 Definition 2.5 Laplace Distriution Let >. We denote y Lap the Laplace distriution such that the proaility density function at z is z 2 exp. Laplace distriution has the following properties. Theorem 2.6 Let > and let γ e a random variale sampled from Lap. Then,. E[γ] =, 2. var[γ] = 2 2, 3. for any λ >, Pr[ γ > λ] = exp λ. Proof: Hence, E[γ] = E[γ 2 ] = = = = = 2 x 2 exp x x x 2 exp x 2 = 2 x 2 exp 2 exp x 2 exp x exp dx = d x + x dx + x x dx = 2 d x x x exp x exp x d x = 2 2 dx = 2 dx + x x 2 exp x x 2 exp dx x x 2 exp x 2 2 exp x = x 2 exp exp Pr[ γ > λ] = Pr[γ > λ] + Pr[γ < λ] = = 2 λ = exp λ x exp x dx x dx = dx x d x = 2 = 2 2 exp x x 2 exp = 2 2 var[γ] = E[γ 2 ] E[γ] 2 = 2 2 = exp x dx = λ λ 2 exp x exp x d x dx 2x exp x dx exp x dx λ dx + x = exp x 2 exp dx x λ 6
7 Note that if is small, the mass is highly concentrated around. Hence, it can only e used to privatize functions with small sensitivity. However, the highly concentrated mass also implies good utility. The following theorem shows that choosing := f is enough to preserve privacy and with high proaility, the additive error incurred y the random noise is small. Theorem 2.7 Let f : D R d e a deterministic function, < < e the privacy parameter and < δ < e the failure proaility. Let γ, γ 2,..., γ d e random variales independently sampled from Lap f. Then, the randomized function f such that f i X := f i x + γ i for all i [d]. preserves -differential privacy, 2. is f ln d δ, δ-useful with respect to f. Proof: Let X D and X 2 D e two neighoring dataase. Let z R d e a vector. We ause the notation a little it and use Pr[ fx = z] and Pr[ fx 2 = z] to denote the density instead of the proaility. Hence, we have Pr[ fx = z] Pr[ fx 2 = z] = Pr[ d f ix + γ i = z i ] Pr[ d f ix 2 + γ i = z i ] = Pr[ d γ i = z i f i X ] Pr[ d γ i = z i f i X 2 ] d = Pr[γ i = z i f i X ] d Pr[γ i = z i f i X 2 ] = d d 2 f exp f ix 2 z i f/ = exp 2 f exp f ix z i f/ d fx 2 z i fx z i f/ f/e d fx fx 2 exp f/ f exp f/ = exp For any measurale suset S R d, Pr[ fx S] = S Pr[ fx = z]dz exp S Pr[ fx 2 = z]dz = Pr[ fx 2 S]. Hence, the privacy guarantee is proved. By Property 3 of Theorem 2.6, we know that for all i [d], Pr[ γ i > f ln d f δ ] = exp ln d f δ / = δ d. Hence, y union ound on i [d], we know that Pr[ i [d] γ i > f ln d δ ] δ. 7
8 Let X D e any dataase. Note that f i X f i X = γ i for all i [d]. Hence, y the union ound, Pr[ i [d], f i X f i X > f ln d δ ] δ, which is equivalent to Pr[ i [d] f i X f i X ] δ. Thus, the utility guarantee is proved. f ln d δ 2.5 Example Consider the previous example where there are n users, each of which holds a private it from {, }, and we would like to calculate how many of them hold a one. In other words, we would like to calculate n X i, where X i is the private it of the i-th user. There is an untrusted aggregator who can calculate the sum of n real numers. To compute the sum, the users need to send information aout their private it to the aggregator, and the aggregator then does the calculation for them. However, since the aggregator might e malicious, the users do not want the aggregator to know their real private its. The protect the users privacy, the information they send to the aggregator should e -differentially private. Note that the sensitivity of each user s private it is one. Hence, a user can first add Laplace noise sampled from Lap to their private it, and then send the noisy it to the aggregator. The aggregator releases the sum of the noisy its as an estimate of the real sum. Oserve that the output is the real sum plus n independent random variales sampled from Lap. It can e shown that the additive error of this estimate is O n with high proaility. This is left as an exercise. 2.6 Exercises. In this question, we derive a measure concentration result for independent random variales drawn from Laplace distriution. We show that with high proaility, the sum of independent Laplace random variales are concentrated around its mean,. We use moment generating functions in a Chernoff-like argument. Let γ, γ 2,..., γ n e n independent random variales, where γ i is sampled from Lap i. a Prove that for each γ i, the moment generating function is E[exphγ i ] =, where h 2 2 i h < i. Show that E[exphγ i ] exp2h 2 2 i, if h < 2i. Hint: for x < 2, we have x + 2x exp2x. c Let M := max i [n] i. Also, let ν Pr[ Y > λ] 2 exp λ2 8ν 2 { n d Suppose < δ < and ν > max 2 i, M δ. e Prove that Pr[ Y > 8 n 2 i ln 2 δ ] δ. n 2 i and < λ < 2 2ν 2 M. Prove that } ln 2 δ. Prove that Pr[ Y > ν 8 ln 2 δ ] 8
1 Approximate Counting by Random Sampling
COMP8601: Advanced Topics in Theoretical Computer Science Lecture 5: More Measure Concentration: Counting DNF Satisfying Assignments, Hoeffding s Inequality Lecturer: Hubert Chan Date: 19 Sep 2013 These
More information1 Dimension Reduction in Euclidean Space
CSIS0351/8601: Randomized Algorithms Lecture 6: Johnson-Lindenstrauss Lemma: Dimension Reduction Lecturer: Hubert Chan Date: 10 Oct 011 hese lecture notes are supplementary materials for the lectures.
More informationPrivacy of Numeric Queries Via Simple Value Perturbation. The Laplace Mechanism
Privacy of Numeric Queries Via Simple Value Perturbation The Laplace Mechanism Differential Privacy A Basic Model Let X represent an abstract data universe and D be a multi-set of elements from X. i.e.
More informationLecture 1 Introduction to Differential Privacy: January 28
Introduction to Differential Privacy CSE711: Topics in Differential Privacy Spring 2016 Lecture 1 Introduction to Differential Privacy: January 28 Lecturer: Marco Gaboardi Scribe: Marco Gaboardi This note
More informationEssential Maths 1. Macquarie University MAFC_Essential_Maths Page 1 of These notes were prepared by Anne Cooper and Catriona March.
Essential Maths 1 The information in this document is the minimum assumed knowledge for students undertaking the Macquarie University Masters of Applied Finance, Graduate Diploma of Applied Finance, and
More informationECEN 689 Special Topics in Data Science for Communications Networks
ECEN 689 Special Topics in Data Science for Communications Networks Nick Duffield Department of Electrical & Computer Engineering Texas A&M University Lecture 3 Estimation and Bounds Estimation from Packet
More informationDifferential Privacy
CS 380S Differential Privacy Vitaly Shmatikov most slides from Adam Smith (Penn State) slide 1 Reading Assignment Dwork. Differential Privacy (invited talk at ICALP 2006). slide 2 Basic Setting DB= x 1
More information1 Randomized Computation
CS 6743 Lecture 17 1 Fall 2007 1 Randomized Computation Why is randomness useful? Imagine you have a stack of bank notes, with very few counterfeit ones. You want to choose a genuine bank note to pay at
More information1 Differential Privacy and Statistical Query Learning
10-806 Foundations of Machine Learning and Data Science Lecturer: Maria-Florina Balcan Lecture 5: December 07, 015 1 Differential Privacy and Statistical Query Learning 1.1 Differential Privacy Suppose
More informationUnderstanding Generalization Error: Bounds and Decompositions
CIS 520: Machine Learning Spring 2018: Lecture 11 Understanding Generalization Error: Bounds and Decompositions Lecturer: Shivani Agarwal Disclaimer: These notes are designed to be a supplement to the
More informationA732: Exercise #7 Maximum Likelihood
A732: Exercise #7 Maximum Likelihood Due: 29 Novemer 2007 Analytic computation of some one-dimensional maximum likelihood estimators (a) Including the normalization, the exponential distriution function
More informationCS Communication Complexity: Applications and New Directions
CS 2429 - Communication Complexity: Applications and New Directions Lecturer: Toniann Pitassi 1 Introduction In this course we will define the basic two-party model of communication, as introduced in the
More informationProofs, Tables and Figures
e-companion to Levi, Perakis and Uichanco: Data-driven Newsvendor Prolem ec Proofs, Tales and Figures In this electronic companion to the paper, we provide proofs of the theorems and lemmas. This companion
More informationLecture 11- Differential Privacy
6.889 New Developments in Cryptography May 3, 2011 Lecture 11- Differential Privacy Lecturer: Salil Vadhan Scribes: Alan Deckelbaum and Emily Shen 1 Introduction In class today (and the next two lectures)
More informationEach element of this set is assigned a probability. There are three basic rules for probabilities:
XIV. BASICS OF ROBABILITY Somewhere out there is a set of all possile event (or all possile sequences of events which I call Ω. This is called a sample space. Out of this we consider susets of events which
More informationANOVA 3/12/2012. Two reasons for using ANOVA. Type I Error and Multiple Tests. Review Independent Samples t test
// ANOVA Lectures - Readings: GW Review Independent Samples t test Placeo Treatment 7 7 7 Mean... Review Independent Samples t test Placeo Treatment 7 7 7 Mean.. t (). p. C. I.: p t tcrit s pt crit s t
More informationLecture 4. P r[x > ce[x]] 1/c. = ap r[x = a] + a>ce[x] P r[x = a]
U.C. Berkeley CS273: Parallel and Distributed Theory Lecture 4 Professor Satish Rao September 7, 2010 Lecturer: Satish Rao Last revised September 13, 2010 Lecture 4 1 Deviation bounds. Deviation bounds
More informationLectures 1&2: Introduction to Secure Computation, Yao s and GMW Protocols
CS 294 Secure Computation January 19, 2016 Lectures 1&2: Introduction to Secure Computation, Yao s and GMW Protocols Instructor: Sanjam Garg Scribe: Pratyush Mishra 1 Introduction Secure multiparty computation
More informationReport on Differential Privacy
Report on Differential Privacy Lembit Valgma Supervised by Vesal Vojdani December 19, 2017 1 Introduction Over the past decade the collection and analysis of personal data has increased a lot. This has
More informationNotes on the Exponential Mechanism. (Differential privacy)
Notes on the Exponential Mechanism. (Differential privacy) Boston University CS 558. Sharon Goldberg. February 22, 2012 1 Description of the mechanism Let D be the domain of input datasets. Let R be the
More informationCS261: A Second Course in Algorithms Lecture #11: Online Learning and the Multiplicative Weights Algorithm
CS61: A Second Course in Algorithms Lecture #11: Online Learning and the Multiplicative Weights Algorithm Tim Roughgarden February 9, 016 1 Online Algorithms This lecture begins the third module of the
More informationOn the Impossibility of Black-Box Truthfulness Without Priors
On the Impossibility of Black-Box Truthfulness Without Priors Nicole Immorlica Brendan Lucier Abstract We consider the problem of converting an arbitrary approximation algorithm for a singleparameter social
More informationPrivacy in Statistical Databases
Privacy in Statistical Databases Individuals x 1 x 2 x n Server/agency ) answers. A queries Users Government, researchers, businesses or) Malicious adversary What information can be released? Two conflicting
More information1 Review of The Learning Setting
COS 5: Theoretical Machine Learning Lecturer: Rob Schapire Lecture #8 Scribe: Changyan Wang February 28, 208 Review of The Learning Setting Last class, we moved beyond the PAC model: in the PAC model we
More informationAnswering Numeric Queries
Answering Numeric Queries The Laplace Distribution: Lap(b) is the probability distribution with p.d.f.: p x b) = 1 x exp 2b b i.e. a symmetric exponential distribution Y Lap b, E Y = b Pr Y t b = e t Answering
More informationDifferential Privacy and its Application in Aggregation
Differential Privacy and its Application in Aggregation Part 1 Differential Privacy presenter: Le Chen Nanyang Technological University lechen0213@gmail.com October 5, 2013 Introduction Outline Introduction
More information1 Systems of Differential Equations
March, 20 7- Systems of Differential Equations Let U e an open suset of R n, I e an open interval in R and : I R n R n e a function from I R n to R n The equation ẋ = ft, x is called a first order ordinary
More informationFORMULATION OF THE LEARNING PROBLEM
FORMULTION OF THE LERNING PROBLEM MIM RGINSKY Now that we have seen an informal statement of the learning problem, as well as acquired some technical tools in the form of concentration inequalities, we
More informationLecture 18: March 15
CS71 Randomness & Computation Spring 018 Instructor: Alistair Sinclair Lecture 18: March 15 Disclaimer: These notes have not been subjected to the usual scrutiny accorded to formal publications. They may
More informationWeak bidders prefer first-price (sealed-bid) auctions. (This holds both ex-ante, and once the bidders have learned their types)
Econ 805 Advanced Micro Theory I Dan Quint Fall 2007 Lecture 9 Oct 4 2007 Last week, we egan relaxing the assumptions of the symmetric independent private values model. We examined private-value auctions
More informationCalibrating Noise to Sensitivity in Private Data Analysis
Calibrating Noise to Sensitivity in Private Data Analysis Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith Mamadou H. Diallo 1 Overview Issues: preserving privacy in statistical databases Assumption:
More informationMinimizing a convex separable exponential function subject to linear equality constraint and bounded variables
Minimizing a convex separale exponential function suect to linear equality constraint and ounded variales Stefan M. Stefanov Department of Mathematics Neofit Rilski South-Western University 2700 Blagoevgrad
More informationLecture 7: Passive Learning
CS 880: Advanced Complexity Theory 2/8/2008 Lecture 7: Passive Learning Instructor: Dieter van Melkebeek Scribe: Tom Watson In the previous lectures, we studied harmonic analysis as a tool for analyzing
More informationSublinear-Time Algorithms
Lecture 20 Sublinear-Time Algorithms Supplemental reading in CLRS: None If we settle for approximations, we can sometimes get much more efficient algorithms In Lecture 8, we saw polynomial-time approximations
More informationStatistics 300B Winter 2018 Final Exam Due 24 Hours after receiving it
Statistics 300B Winter 08 Final Exam Due 4 Hours after receiving it Directions: This test is open book and open internet, but must be done without consulting other students. Any consultation of other students
More information20.1 2SAT. CS125 Lecture 20 Fall 2016
CS125 Lecture 20 Fall 2016 20.1 2SAT We show yet another possible way to solve the 2SAT problem. Recall that the input to 2SAT is a logical expression that is the conunction (AND) of a set of clauses,
More informationLecture 12: Grover s Algorithm
CPSC 519/619: Quantum Computation John Watrous, University of Calgary Lecture 12: Grover s Algorithm March 7, 2006 We have completed our study of Shor s factoring algorithm. The asic technique ehind Shor
More information{X i } realize. n i=1 X i. Note that again X is a random variable. If we are to
3 Convergence This topic will overview a variety of extremely powerful analysis results that span statistics, estimation theorem, and big data. It provides a framework to think about how to aggregate more
More informationCIS 700 Differential Privacy in Game Theory and Mechanism Design January 17, Lecture 1
CIS 700 Differential Privacy in Game Theory and Mechanism Design January 17, 2014 Lecture 1 Lecturer: Aaron Roth Scribe: Aaron Roth Intro to Differential Privacy for Game Theorists, and Digital Goods Auctions.
More informationNotes on Property-Preserving Encryption
Notes on Property-Preserving Encryption The first type of specialized encryption scheme that can be used in secure outsourced storage we will look at is property-preserving encryption. This is encryption
More informationEVALUATIONS OF EXPECTED GENERALIZED ORDER STATISTICS IN VARIOUS SCALE UNITS
APPLICATIONES MATHEMATICAE 9,3 (), pp. 85 95 Erhard Cramer (Oldenurg) Udo Kamps (Oldenurg) Tomasz Rychlik (Toruń) EVALUATIONS OF EXPECTED GENERALIZED ORDER STATISTICS IN VARIOUS SCALE UNITS Astract. We
More informationCSCI 5520: Foundations of Data Privacy Lecture 5 The Chinese University of Hong Kong, Spring February 2015
CSCI 5520: Foundations of Data Privacy Lecture 5 The Chinese University of Hong Kong, Spring 2015 3 February 2015 The interactive online mechanism that we discussed in the last two lectures allows efficient
More informationLecture 20: Introduction to Differential Privacy
6.885: Advanced Topics in Data Processing Fall 2013 Stephen Tu Lecture 20: Introduction to Differential Privacy 1 Overview This lecture aims to provide a very broad introduction to the topic of differential
More informationMath 5051 Measure Theory and Functional Analysis I Homework Assignment 3
Math 551 Measure Theory and Functional Analysis I Homework Assignment 3 Prof. Wickerhauser Due Monday, October 12th, 215 Please do Exercises 3*, 4, 5, 6, 8*, 11*, 17, 2, 21, 22, 27*. Exercises marked with
More informationProblem Set 0 Solutions
CS446: Machine Learning Spring 2017 Problem Set 0 Solutions Handed Out: January 25 th, 2017 Handed In: NONE 1. [Probability] Assume that the probability of obtaining heads when tossing a coin is λ. a.
More informationHomework 4 Solutions
CS 174: Combinatorics and Discrete Probability Fall 01 Homework 4 Solutions Problem 1. (Exercise 3.4 from MU 5 points) Recall the randomized algorithm discussed in class for finding the median of a set
More informationarxiv: v1 [cs.fl] 24 Nov 2017
(Biased) Majority Rule Cellular Automata Bernd Gärtner and Ahad N. Zehmakan Department of Computer Science, ETH Zurich arxiv:1711.1090v1 [cs.fl] 4 Nov 017 Astract Consider a graph G = (V, E) and a random
More informationIntroduction to Algorithms / Algorithms I Lecturer: Michael Dinitz Topic: Intro to Learning Theory Date: 12/8/16
600.463 Introduction to Algorithms / Algorithms I Lecturer: Michael Dinitz Topic: Intro to Learning Theory Date: 12/8/16 25.1 Introduction Today we re going to talk about machine learning, but from an
More informationSample Solutions from the Student Solution Manual
1 Sample Solutions from the Student Solution Manual 1213 If all the entries are, then the matrix is certainly not invertile; if you multiply the matrix y anything, you get the matrix, not the identity
More informationCSE 525 Randomized Algorithms & Probabilistic Analysis Spring Lecture 3: April 9
CSE 55 Randomized Algorithms & Probabilistic Analysis Spring 01 Lecture : April 9 Lecturer: Anna Karlin Scribe: Tyler Rigsby & John MacKinnon.1 Kinds of randomization in algorithms So far in our discussion
More informationCMPUT651: Differential Privacy
CMPUT65: Differential Privacy Homework assignment # 2 Due date: Apr. 3rd, 208 Discussion and the exchange of ideas are essential to doing academic work. For assignments in this course, you are encouraged
More informationMachine Learning. Lecture 9: Learning Theory. Feng Li.
Machine Learning Lecture 9: Learning Theory Feng Li fli@sdu.edu.cn https://funglee.github.io School of Computer Science and Technology Shandong University Fall 2018 Why Learning Theory How can we tell
More informationDivide-and-Conquer. Reading: CLRS Sections 2.3, 4.1, 4.2, 4.3, 28.2, CSE 6331 Algorithms Steve Lai
Divide-and-Conquer Reading: CLRS Sections 2.3, 4.1, 4.2, 4.3, 28.2, 33.4. CSE 6331 Algorithms Steve Lai Divide and Conquer Given an instance x of a prolem, the method works as follows: divide-and-conquer
More informationProblem 1: (Chernoff Bounds via Negative Dependence - from MU Ex 5.15)
Problem 1: Chernoff Bounds via Negative Dependence - from MU Ex 5.15) While deriving lower bounds on the load of the maximum loaded bin when n balls are thrown in n bins, we saw the use of negative dependence.
More informationLecture 4: Two-point Sampling, Coupon Collector s problem
Randomized Algorithms Lecture 4: Two-point Sampling, Coupon Collector s problem Sotiris Nikoletseas Associate Professor CEID - ETY Course 2013-2014 Sotiris Nikoletseas, Associate Professor Randomized Algorithms
More informationMotivation: Can the equations of physics be derived from information-theoretic principles?
10. Physics from Fisher Information. Motivation: Can the equations of physics e derived from information-theoretic principles? I. Fisher Information. Task: To otain a measure of the accuracy of estimated
More information2 How many distinct elements are in a stream?
Dealing with Massive Data January 31, 2011 Lecture 2: Distinct Element Counting Lecturer: Sergei Vassilvitskii Scribe:Ido Rosen & Yoonji Shin 1 Introduction We begin by defining the stream formally. Definition
More information1 A Lower Bound on Sample Complexity
COS 511: Theoretical Machine Learning Lecturer: Rob Schapire Lecture #7 Scribe: Chee Wei Tan February 25, 2008 1 A Lower Bound on Sample Complexity In the last lecture, we stopped at the lower bound on
More informationLecture 1 Measure concentration
CSE 29: Learning Theory Fall 2006 Lecture Measure concentration Lecturer: Sanjoy Dasgupta Scribe: Nakul Verma, Aaron Arvey, and Paul Ruvolo. Concentration of measure: examples We start with some examples
More information1 Distributional problems
CSCI 5170: Computational Complexity Lecture 6 The Chinese University of Hong Kong, Spring 2016 23 February 2016 The theory of NP-completeness has been applied to explain why brute-force search is essentially
More informationActive Learning: Disagreement Coefficient
Advanced Course in Machine Learning Spring 2010 Active Learning: Disagreement Coefficient Handouts are jointly prepared by Shie Mannor and Shai Shalev-Shwartz In previous lectures we saw examples in which
More informationHeavy Hitters and the Structure of Local Privacy
Heavy Hitters and the Structure of Local Privacy Mark Bun Jelani Nelson Uri Stemmer November 3, 207 Abstract We present a new locally differentially private algorithm for the heavy hitters problem which
More informationA = A U. U [n] P(A U ). n 1. 2 k(n k). k. k=1
Lecture I jacques@ucsd.edu Notation: Throughout, P denotes probability and E denotes expectation. Denote (X) (r) = X(X 1)... (X r + 1) and let G n,p denote the Erdős-Rényi model of random graphs. 10 Random
More informationQuantum query complexity of entropy estimation
Quantum query complexity of entropy estimation Xiaodi Wu QuICS, University of Maryland MSR Redmond, July 19th, 2017 J o i n t C e n t e r f o r Quantum Information and Computer Science Outline Motivation
More informationAt first numbers were used only for counting, and 1, 2, 3,... were all that was needed. These are called positive integers.
1 Numers One thread in the history of mathematics has een the extension of what is meant y a numer. This has led to the invention of new symols and techniques of calculation. When you have completed this
More informationRandomized algorithm
Tutorial 4 Joyce 2009-11-24 Outline Solution to Midterm Question 1 Question 2 Question 1 Question 2 Question 3 Question 4 Question 5 Solution to Midterm Solution to Midterm Solution to Midterm Question
More informationLecture 4: Codes based on Concatenation
Lecture 4: Codes based on Concatenation Error-Correcting Codes (Spring 206) Rutgers University Swastik Kopparty Scribe: Aditya Potukuchi and Meng-Tsung Tsai Overview In the last lecture, we studied codes
More informationLectures One Way Permutations, Goldreich Levin Theorem, Commitments
Lectures 11 12 - One Way Permutations, Goldreich Levin Theorem, Commitments Boaz Barak March 10, 2010 From time immemorial, humanity has gotten frequent, often cruel, reminders that many things are easier
More informationLecture 23: Alternation vs. Counting
CS 710: Complexity Theory 4/13/010 Lecture 3: Alternation vs. Counting Instructor: Dieter van Melkebeek Scribe: Jeff Kinne & Mushfeq Khan We introduced counting complexity classes in the previous lecture
More informationNotes on Complexity Theory Last updated: November, Lecture 10
Notes on Complexity Theory Last updated: November, 2015 Lecture 10 Notes by Jonathan Katz, lightly edited by Dov Gordon. 1 Randomized Time Complexity 1.1 How Large is BPP? We know that P ZPP = RP corp
More informationLecture 9: March 26, 2014
COMS 6998-3: Sub-Linear Algorithms in Learning and Testing Lecturer: Rocco Servedio Lecture 9: March 26, 204 Spring 204 Scriber: Keith Nichols Overview. Last Time Finished analysis of O ( n ɛ ) -query
More informationLecture 5: Probabilistic tools and Applications II
T-79.7003: Graphs and Networks Fall 2013 Lecture 5: Probabilistic tools and Applications II Lecturer: Charalampos E. Tsourakakis Oct. 11, 2013 5.1 Overview In the first part of today s lecture we will
More information1 Introduction (January 21)
CS 97: Concrete Models of Computation Spring Introduction (January ). Deterministic Complexity Consider a monotonically nondecreasing function f : {,,..., n} {, }, where f() = and f(n) =. We call f a step
More information1 Large Deviations. Korea Lectures June 2017 Joel Spencer Tuesday Lecture III
Korea Lectures June 017 Joel Spencer Tuesday Lecture III 1 Large Deviations We analyze how random variables behave asymptotically Initially we consider the sum of random variables that take values of 1
More information10 Lorentz Group and Special Relativity
Physics 129 Lecture 16 Caltech, 02/27/18 Reference: Jones, Groups, Representations, and Physics, Chapter 10. 10 Lorentz Group and Special Relativity Special relativity says, physics laws should look the
More informationWhy (Bayesian) inference makes (Differential) privacy easy
Why (Bayesian) inference makes (Differential) privacy easy Christos Dimitrakakis 1 Blaine Nelson 2,3 Aikaterini Mitrokotsa 1 Benjamin Rubinstein 4 Zuhe Zhang 4 1 Chalmers university of Technology 2 University
More informationNotes for Lecture A can repeat step 3 as many times as it wishes. We will charge A one unit of time for every time it repeats step 3.
COS 533: Advanced Cryptography Lecture 2 (September 18, 2017) Lecturer: Mark Zhandry Princeton University Scribe: Mark Zhandry Notes for Lecture 2 1 Last Time Last time, we defined formally what an encryption
More informationSTRONG NORMALITY AND GENERALIZED COPELAND ERDŐS NUMBERS
#A INTEGERS 6 (206) STRONG NORMALITY AND GENERALIZED COPELAND ERDŐS NUMBERS Elliot Catt School of Mathematical and Physical Sciences, The University of Newcastle, Callaghan, New South Wales, Australia
More informationThe Fitness Level Method with Tail Bounds
The Fitness Level Method with Tail Bounds Carsten Witt DTU Compute Technical University of Denmark 2800 Kgs. Lyngby Denmark arxiv:307.4274v [cs.ne] 6 Jul 203 July 7, 203 Abstract The fitness-level method,
More information[Title removed for anonymity]
[Title removed for anonymity] Graham Cormode graham@research.att.com Magda Procopiuc(AT&T) Divesh Srivastava(AT&T) Thanh Tran (UMass Amherst) 1 Introduction Privacy is a common theme in public discourse
More informationSection 2.1: Reduce Rational Expressions
CHAPTER Section.: Reduce Rational Expressions Section.: Reduce Rational Expressions Ojective: Reduce rational expressions y dividing out common factors. A rational expression is a quotient of polynomials.
More informationGeneralization Bounds in Machine Learning. Presented by: Afshin Rostamizadeh
Generalization Bounds in Machine Learning Presented by: Afshin Rostamizadeh Outline Introduction to generalization bounds. Examples: VC-bounds Covering Number bounds Rademacher bounds Stability bounds
More informationHandout 5. α a1 a n. }, where. xi if a i = 1 1 if a i = 0.
Notes on Complexity Theory Last updated: October, 2005 Jonathan Katz Handout 5 1 An Improved Upper-Bound on Circuit Size Here we show the result promised in the previous lecture regarding an upper-bound
More informationMachine Learning Theory Lecture 4
Machine Learning Theory Lecture 4 Nicholas Harvey October 9, 018 1 Basic Probability One of the first concentration bounds that you learn in probability theory is Markov s inequality. It bounds the right-tail
More informationLecture notes on OPP algorithms [Preliminary Draft]
Lecture notes on OPP algorithms [Preliminary Draft] Jesper Nederlof June 13, 2016 These lecture notes were quickly assembled and probably contain many errors. Use at your own risk! Moreover, especially
More informationMachine Learning
Machine Learning 10-601 Tom M. Mitchell Machine Learning Department Carnegie Mellon University October 11, 2012 Today: Computational Learning Theory Probably Approximately Coorrect (PAC) learning theorem
More informationThe expansion of random regular graphs
The expansion of random regular graphs David Ellis Introduction Our aim is now to show that for any d 3, almost all d-regular graphs on {1, 2,..., n} have edge-expansion ratio at least c d d (if nd is
More informationLecture 4: Hashing and Streaming Algorithms
CSE 521: Design and Analysis of Algorithms I Winter 2017 Lecture 4: Hashing and Streaming Algorithms Lecturer: Shayan Oveis Gharan 01/18/2017 Scribe: Yuqing Ai Disclaimer: These notes have not been subjected
More informationLattice Cryptography
CSE 06A: Lattice Algorithms and Applications Winter 01 Instructor: Daniele Micciancio Lattice Cryptography UCSD CSE Many problems on point lattices are computationally hard. One of the most important hard
More informationAnswering Many Queries with Differential Privacy
6.889 New Developments in Cryptography May 6, 2011 Answering Many Queries with Differential Privacy Instructors: Shafi Goldwasser, Yael Kalai, Leo Reyzin, Boaz Barak, and Salil Vadhan Lecturer: Jonathan
More informationRepresentation theory of SU(2), density operators, purification Michael Walter, University of Amsterdam
Symmetry and Quantum Information Feruary 6, 018 Representation theory of S(), density operators, purification Lecture 7 Michael Walter, niversity of Amsterdam Last week, we learned the asic concepts of
More informationCompactness vs Collusion Resistance in Functional Encryption
Compactness vs Collusion Resistance in Functional Encryption Baiyu Li Daniele Micciancio April 10, 2017 Astract We present two general constructions that can e used to comine any two functional encryption
More informationLecture 14: Secure Multiparty Computation
600.641 Special Topics in Theoretical Cryptography 3/20/2007 Lecture 14: Secure Multiparty Computation Instructor: Susan Hohenberger Scribe: Adam McKibben 1 Overview Suppose a group of people want to determine
More informationUniversity of Regina. Lecture Notes. Michael Kozdron
University of Regina Statistics 252 Mathematical Statistics Lecture Notes Winter 2005 Michael Kozdron kozdron@math.uregina.ca www.math.uregina.ca/ kozdron Contents 1 The Basic Idea of Statistics: Estimating
More informationCS261: A Second Course in Algorithms Lecture #18: Five Essential Tools for the Analysis of Randomized Algorithms
CS261: A Second Course in Algorithms Lecture #18: Five Essential Tools for the Analysis of Randomized Algorithms Tim Roughgarden March 3, 2016 1 Preamble In CS109 and CS161, you learned some tricks of
More information1. Implement AdaBoost with boosting stumps and apply the algorithm to the. Solution:
Mehryar Mohri Foundations of Machine Learning Courant Institute of Mathematical Sciences Homework assignment 3 October 31, 2016 Due: A. November 11, 2016; B. November 22, 2016 A. Boosting 1. Implement
More informationStephen F Austin. Exponents and Logarithms. chapter 3
chapter 3 Starry Night was painted by Vincent Van Gogh in 1889. The brightness of a star as seen from Earth is measured using a logarithmic scale. Exponents and Logarithms This chapter focuses on understanding
More informationLecture 11: Key Agreement
Introduction to Cryptography 02/22/2018 Lecture 11: Key Agreement Instructor: Vipul Goyal Scribe: Francisco Maturana 1 Hardness Assumptions In order to prove the security of cryptographic primitives, we
More informationExponential Tail Bounds
Exponential Tail Bounds Mathias Winther Madsen January 2, 205 Here s a warm-up problem to get you started: Problem You enter the casino with 00 chips and start playing a game in which you double your capital
More informationExpectation of geometric distribution
Expectation of geometric distribution What is the probability that X is finite? Can now compute E(X): Σ k=1f X (k) = Σ k=1(1 p) k 1 p = pσ j=0(1 p) j = p 1 1 (1 p) = 1 E(X) = Σ k=1k (1 p) k 1 p = p [ Σ
More information