ON THE ADVERSARIAL ROBUSTNESS OF ROBUST ESTIMATORS

Transcription

1 O THE ADVERSARIAL ROBUSTESS OF ROBUST ESTIMATORS BY ERHA BAYAKTAR,, LIFEG LAI, University of Michigan and University of California, Davis Motivated by recent data analytics alications, we study the adversarial robustness of robust estimators. Instead of assuming that only a fraction of the data oints are outliers as considered in the classic robust estimation setu, in this aer, we consider an adversarial setu in which an attacker can observe the whole dataset and can modify all data samles in an adversarial manner so as to maximize the estimation error caused by his attack. We characterize the attacker s otimal attack strategy, and further introduce adversarial influence function (AIF) to quantify an estimator s sensitivity to such adversarial attacks. We rovide an aroach to characterize AIF for any given robust estimator, and then design otimal estimator that minimizes AIF, which imlies it is least sensitive to adversarial attacks and hence is most robust against adversarial attacks. From this characterization, we identify a tradeoff between AIF (i.e., robustness against adversarial attack) and influence function, a quantity used in classic robust estimators to measure robustness against outliers, and design estimators that strike a desirable tradeoff between these two quantities.. Introduction. Robust estimation is a classic toic that addresses the outlier or model uncertainty issues. In the existing setu, a certain ercentage of the data oints are assumed to be outliers. Various concets such as influence function (IF), breakdown oint, and change of variance etc were develoed to quantify the robustness of estimators against the resence of outliers, lease see [, 2, 3] and references therein for details. These concets are very useful for the classic setu where a fraction (u to 50%) of data oints are outliers while the remaining data come from the true distribution. In this aer, motivated by recent interest in data analytics, we address the issue of adversarial robustness. In a tyical data analytics setu, a dataset is stored in a database. If an attacker has access to the database, he can modify all data oints (i.e., u to 00%) in an adversarial manner, and hence the existing results on robust statistics are not directly alicable anymore. This scenario also arises in the adversarial examle henomena in dee neural networks that have attracted signif- The work of E. Bayraktar was suorted in art by the ational Science Foundation under grant DMS-6370 and by the Susan M. Smith Professorshi. The work of L. Lai was suorted by the ational Science Foundation under grants CCF and ECCS MSC 200 subject classifications: Primary 62F35; secondary 62F0,62F2 Keywords and hrases: Robust estimators, adversarial robustness, M-estimator, non-convex otimization.

2 2 E. BAYAKTAR AD L. LAI icant recent research interests [4, 5, 6]. In the adversarial examle in dee neural networks, by making small but carefully chosen changes on the image, the attacker can mislead neural network to make wrong decisions, even though a human will hardly notice changes on the modified image. Certainly, if the attacker can modify all data and no further restrictions on attacker s caability are imosed, then no meaningful estimator can be constructed (this can be viewed as 00% of the data are modified in the classic setu). In this aer, we investigate the scenario that the total amount of change measured by l norm is limited, and we will study how these quantities will affect the estimation erformance. This tye of constraints are reasonable and are motivated by real life examles. For examle, in generating adversarial examles in images [4], the total distortion should be limited, otherwise human eyes will be able to detect such changes. Towards this goal, we introduce the concet of adversarial influence function (AIF) to quantify how sensitive an estimator is to adversarial attacks. We first focus on the scenario with a given data set. For this scenario, we characterize the otimal attack vector that the attacker, who observes the whole data set, can emloy to maximize the estimation error. Using this characterization, we can then analyze AIF of any given estimator. This analysis enables us to design estimators that are robust to adversarial attacks. In articular, from the estimator s ersective, one would like to design an estimator that minimizes AIF, which imlies that such an estimator is least sensitive to adversarial attacks and hence is most robust against adversarial attacks. We derive universal lower bounds on AIF and characterize the conditions under which an estimator can achieve this lower bound (and hence is most robustness against adversarial attacks). We then illustrate these results for two secific models: location estimators and scale estimators. With the results in the given samle scenario, we then extend our study to the oulation scenario, in which we investigate the behavior of AIF as the number of samles increases. For this case, we identify a tradeoff between robustness against adversarial attacks vs robustness against outliers. In articular, we first characterize the otimal estimator that minimizes AIF. However, the estimator that minimizes AIF has a oor erformance in term of IF [3, 7], a quantity that measures robustness against outliers. Realizing this fact, we then formulate otimization roblems to design estimators that strike a desirable tradeoff between AIF (i.e., robustness against adversarial attack) and IF (i.e., robustness against outliers). Using tools from calculus of variations [8, 9], we are able to exloit the unique structure of our roblems and obtain analytical form of the otimal solution. The obtained solution share similar interretation as classic robust estimators that it will carefully trim data oints that are from the distribution tails. However, the detailed form and thresholds are determined by different criteria. In the above discussion, we mainly focus on a class of widely used robust esti-

3 ADVERSARIAL ROBUSTESS OF ROBUST ESTIMATORS 3 mators: M-estimator. However, the develoed tools and analysis can be extended to analyze other tyes of robust estimators. In this aer, we will use L-estimator as an examle to discuss how to extend the analysis to other tyes of estimators. Our aer is related to a growing list of recent work on adversarial machine learning. Here we give several examles on data oisoning attack that is related to our work. For examle, [0] considers an adversarial rincial comonent analysis (PCA) roblem in which an attacker adds an extra data oint in an adversarial manner so as to maximize the error of subsace estimated by PCA. [] investigates data oisoning attack in regression roblems, in which the attacker adds data oints to the training dataset with the goal of introducing errors into or guiding the results of regression models. [2] studies an attack that inserts carefully chosen data oints to the training set for suort vector machine. [3] considers learning roblems from untrusted data. In articular, under the assumtion that at least α ercent of data oints are drawn from a distribution of interest, [3] considers two frameworks: ) list-decodable learning, whose goal is to return a list of answers, with the guarantee that at least one of them is accurate; and 2) semi-verified learning, in which one has a small dataset of trusted data that can be leveraged to enable the accurate extraction of information from a much larger but untrusted dataset. A major difference between our work and these interesting work is that the existing work assume that a certain ercentage of data oints are not comromised, while in our work all data oints could be comromised. The remainder of the aer is organized as follows. In Section 2, we introduce our roblem formulation. In Section 3, we introduce the necessary background. In Section 4, we investigate AIF for the given samle scenario. In Section 5, we consider the oulation scenario. We extend the study to L-estimator in Section 6. umerical examles are given in Section 7. Finally, we offer concluding remarks in Section Model. We consider an adversarially robust arameter estimation roblem in which the adversary has access to the whole dataset. In articular, we have a given data set x = {x,, x }, in which x n are i.i.d realizations of random variable X that has cumulative density function (cdf) F θ (x) with unknown arameter θ. We will use f θ (x) to denote the corresonding robability density function (df). From this given data set, we would like to estimate the unknown arameter θ. However, as the adversary has access to the whole dataset, it will modify the data to x = x + x := {x + x,, x + x }, in which x = { x,, x } is the attack vector chosen by the adversary after observing x. We will discuss the attacker s otimal attack strategy in choosing x in the sequel. In the classic robust estimation setu, it is tyically assume that some ercentage (u to 50%)

4 4 E. BAYAKTAR AD L. LAI of the data oints are outliers, that is some entries in x are nonzero while the remainders are zero. In this work, we consider the case where the attacker can modify all data oints, which is a more suitable setu for recent data analytical alications. However, certain restrictions need to be ut on x, otherwise the estimation roblem will not be meaningful. In this aer, we assume that x η, in which is the l norm. The normalization factor imlies that the erdimension change (on average) is uer-bound by η. As mentioned in the introduction, this tye of constraints are reasonable and are motivated by real life examles. The classic setu can be viewed as a secial case of our formulation by letting 0, i.e., the classic setu has constraint on the total number of data oints that the attacker can modify. Following notation used in robust statistics [2, 3], we will use T (x) to denote an estimator. For a given estimator T, we would like to characterize how sensitive the estimator is with resect to the adversarial attack. In this aer, we consider a scenario where the goal of the attacker is to maximize the estimation error caused by the attack. In articular, the attacker aims to choose x by solving the following otimization roblem (2.) max x s.t. T (x + x) T (x), x η. We use T (x) to denote the otimal value obtained from the otimization roblem (2.), and define the adversarial influence function (AIF) of estimator T at x under l norm constraint as T (x) AIF(T, x, ) = lim. η 0 η This quantity, a generalization of the concet of IF used in classic robust estimation (we will briefly review IF in Section 3), quantifies the asymtotic rate at which the attacker can introduce estimation error through its attack. From the defender s ersective, the smaller AIF is, the more robust the estimator is. In this aer, building on the characterization of AIF(T, x, ), we will characterize the otimal estimator T, among a certain class of estimators T, that minimizes AIF(T, x, ). In articular, we will investigate min AIF(T, x, ). T T

5 ADVERSARIAL ROBUSTESS OF ROBUST ESTIMATORS 5 We will show that, for certain class of T, the otimal T is indeendent of x and, which is a very desirable roerty. ote that AIF(T, x, ) deends on the data realization x. Based on the characterization of AIF for a given data realization x of length, we will then study the oulation version of AIF where each entry of X = {X,, X } is i.i.d generated by F θ. We will examine the behavior of AIF(T, X, ) as increases. Following the convention in robust statistics, we will assume that there exists a functional T such that (2.2) T (X) T (F θ ) in robability as. We will see that for a large class of estimators AIF(T, X, ) has a well-defined limit as. We will use AIF(T, F θ, ) to denote this limit when it exists. Similarly, from the defense s ersective, we would like to design an estimator that is least sensitive to the adversarial attack. Again, we will characterize the otimal estimator T, among a certain class of estimators T, that minimizes AIF(T, F θ, ). That is, for a certain class of estimators T, we will solve (2.3) min AIF(T, F θ, ). T T It will be clear in the sequel that the solution to the otimization roblem (2.3), even though is robust against adversarial attacks, has oor erformance in guarding against outliers. This motivates us to design estimators that strike a desirable tradeoff between these two robustness measures. In articular, we will solve (2.3) with an additional constraint on IF. We will need to use tools from calculus of variations for this urose. 3. Background. In this section, we briefly review results from classic robust estimator literature that are closely related to our study. 3.. Influence Function (IF). As mentioned above, in the classic robust estimation setu, it is assumed that a fraction η of data oints are outliers, while the remainder of data oints are generated from the true distribution F θ. For a given estimator T, the concet of IF introduced by Hamer [7] is defined T (( η)f θ + ηδ x ) T (F θ ) IF(x, T, F θ ) = lim. η 0 η In this definition, δ x is a distribution that uts mass at oint x, T (F θ ), introduced in (2.2), is the obtained estimate when all data oints are generated i.i.d from F θ, and T (( η)f θ + ηδ x ) is the obtained estimate when η fraction of data oints

6 6 E. BAYAKTAR AD L. LAI are generated i.i.d from F θ while η fraction of the data oints are at x. Hence, IF(x, T, F θ ) measures the asymtotic influence of having outliers at oint x as η 0. To measure the influence of the worst outliers, [7] then further introduced the concet of gross-error sensitivity of T by taking su over the absolute value of IF(x, T, F θ ): γ (T, F θ ) = su IF(x, T, F θ ). x Intuitively seaking, γ (T, F θ ) can be viewed as the solution of our roblem setu for the secial case of = 0. The values of IF(x, T, F θ ) and γ (T, F θ ) have been characterized for various class of estimators. Furthermore, under certain conditions, otimal estimator T that minimizes these quantities have been established. Some of these results will be introduced in later sections. More details can be found in [2, 3] M-Estimator. In this aer, we will mainly focus on a class of commonly used estimator in robust statistic: M-estimator [], in which one obtains an estimate T (x) of θ by solving (3.) ψ(x n, T ) = 0. n= Here ψ(x n, θ) is a function of data x n and arameter θ to be estimated. Different choices of ψ lead to different robust estimators. For examle, the most likely estimator (MLE) can be obtained by setting ψ = f θ /f θ. As the form of ψ determines T, in the remainder of the aer, we will use ψ and T interchangeably. For examle, we will denote AIF(T, x, ) as AIF(ψ, x, ). Similarly, we will denote IF(x, T, F θ ) as IF(x, ψ, F θ ). It is tyically assumed that ψ(x, θ) is continuous and almost everywhere differentiable. This assumtion is valid for all ψ s that are commonly used. It is also tyically assume the estimator is Fisher consistent [3]: (3.2) E Fθ [ψ(x, θ)] = 0, in which E Fθ means exectation under F θ. Intuitively seaking, this imlies that the true arameter θ is the solution of the M-estimator if there are increasingly more i.i.d. data oints generated from F θ. For M-estimator, IF(x, ψ, F θ ) was shown to be see (2.3.5) of [3]. ψ(x, T (F θ )) IF(x, ψ, F θ ) = θ [ψ(y, θ)] θ=t (F θ )df θ (y),

7 ADVERSARIAL ROBUSTESS OF ROBUST ESTIMATORS 7 4. The Fixed Samle Case. In this section, we focus on analyzing AIF(ψ, x, ) for a given dataset x. We will extend the study to the oulation case and analyze AIF(ψ, F θ, ) in Section General ψ. We will first characterize AIF(ψ, x, ) for general ψ, and will then secialize the results to secific roblems in later sections. For any given ψ that is continuous and almost everywhere differentiable, we have the following theorem that characterizes AIF(ψ, x, ). THEOREM 4.. When =, AIF(ψ, x, ) = x [ψ] x=x n,θ=t, θ [ψ] x=x n,θ=t n= where (4.) n = arg max n x [ψ] x=x n,θ=t. For >, we have AIF(ψ, x, ) = ( n= x [ψ] x=x n,θ=t n= ). θ [ψ] x=x n,θ=t PROOF. Please see Aendix A for detailed roof. From Theorem 4., we can characterize the form of ψ that leads to the smallest AIF, i.e., the most robust M-estimator against adversarial attacks. COROLLARY 4.. AIF(ψ, x, ) n= n= x [ψ] x=x n,θ=t, θ [ψ] x=x n,θ=t and the equality holds when x [ψ] x=x,θ=t = = x [ψ] x=x,θ=t.

8 8 E. BAYAKTAR AD L. LAI PROOF. For >, it is easy to check that x ( )/ is a concave function when x 0. Hence, using Jensen s inequality, we have ( x [ψ] x=x n,θ=t n= ) x [ψ] x=x n,θ=t, and the equality holds when x [ψ] x=x n,θ=t is a constant with resect to n. This corollary imlies that, from defender s ersective, we should design ψ(x, θ) such that x [ψ] is constant in x. It is also interesting that, this result holds for any value of. And hence we can design an estimator without knowledge about which constraint the attacker is using Secific Estimators. To illustrate the results obtained above, we secialize results to location estimators and scale estimators Location Estimator. For location estimator models, F θ (x) = F 0 (x θ), and hence it is natural to use ψ(x, θ) = ψ(x θ), see [2, 3]. For this model, it is easy to check that x [ψ] x=x n,θ=t = ψ (x n T ), θ [ψ] x=x n,θ=t = ψ (x n T ). Plugging these two equations in the AIF exressions in Theorem 4., for the case with =, we have ψ (x n T ) AIF(ψ, x, ) =, ψ (x n T ) n= for which the equality holds when ψ (x n T ) is a constant with resect to n. For the case with >, we have (4.2) AIF(ψ, x, ) = (a) ( n= ψ (x n T ) n= ψ (x n T ) n= ψ (x n T ), ψ (x n T ) n= n= )

9 ADVERSARIAL ROBUSTESS OF ROBUST ESTIMATORS 9 in which (a) is due to Jensen s inequality. Both inequalities will hold if ψ (x n T ) is a constant in n. EXAMPLE 4.. Consider an estimator with ψ(x n T ) = x n T. This estimator is simly the emirical samle mean. It is easy to see that ψ (x) is a constant in n, which imlies that this choice of ψ has AIF(ψ, x, ) =. It achieves the lower bound established above, regardless of the value of x and. Hence, it is the most robust estimator against adversarial attacks. However, as we will discuss in Section 5, this choice of ψ is not robust against outliers. In Section 5, we will design estimators that strike a desirable balance between robustness against outliers and robustness against adversarial attacks. EXAMPLE 4.2. Consider the Huber estimator [] with ψ(x n T ) = min{b, max{x n T, b}}, arameterized by a arameter 0 < b <. Using (4.2), it is easy to check that AIF(ψ, x, ) = /β, in which β is the ercentage of oints in x such that x n T < b. It is clear that Huber estimator, while being more robust against outliers [2], is less robust against than adversarial attacks than the emirical mean estimator Scale Estimator. The scale model [2, 3] is given by F θ (x) = F (x/θ), and it is tyical to consider ψ(x, θ) = ψ(x/θ). It is easy to check that for ψ with this form, we have x [ψ] x=x n,θ=t = ψ (x n /T ), T θ [ψ] x=x n,θ=t = x nψ (x n /T ). Using Theorem 4., for the case with =, we obtain ψ (x n /T ) T ψ (x n /T ) (4.3) AIF(ψ, x, ) = =, x n /T ψ (x n /T ) n= in which n is defined in (4.). x nψ (x n/t ) T 2 T 2 n=

10 0 E. BAYAKTAR AD L. LAI When >, we have AIF(ψ, x, ) = (4.4) = ( ( ψ (x n/t ) T n= n= n= n= ) x nψ (x n/t ) T 2 ) ψ (x n /T ) x n /T ψ (x n /T ). EXAMPLE 4.3. Consider MLE for variance of zero-mean Gaussian random variables, which corresonds to ψ(x) = x(φ (x)/φ(x)) = x 2. Here, φ(x) is the df of zero mean variance one Gaussian random variable. For this choice of ψ, we have T = n= x2 n and ψ (x n /T ) = 2x n /T. Plugging these values into (4.3), we obtain Using (4.4), we have AIF(ψ, x, ) = x n. ( ) AIF(ψ, x, ) = x n, from which we know that when = 2, AIF(ψ, x, 2) = T = n= x2 n. n= 5. Poulation Case. With the results on the fixed dataset case, we now consider the oulation version where X n are i.i.d from F θ, and analyze the behavior of AIF as. Following the convention in classic robust statistic literature, we will focus on the case in which the estimator is Fisher consistent as defined a.s. in (3.2). Under a broad range of regularity conditions, [2] shows that T θ. As a result, following similar arguments in Proosition 3 of [4], as in Theorem 4., we have, (5.) AIF(ψ, x, ) = a.s. x [ψ] x=x n,θ=t n= θ [ψ] x=x n,θ=t max x [ E Fθ θ [ψ](x, θ)] := AIF(ψ, F θ, ). x [ψ](x, θ)

11 For >, we have ADVERSARIAL ROBUSTESS OF ROBUST ESTIMATORS (5.2) AIF(ψ, x, ) = ( a.s. x n [ψ] x=xn,θ=t θ [ψ] x=x n,θ=t n= n= ( [ E Fθ x [ψ](x, θ) [ EFθ θ [ψ](x, θ)] ) ]) := AIF(ψ, F θ, ). 5.. Location Estimator. We now secialize the results to the location model mentioned above. We will first characterize ψ that minimizes AIF(ψ, F θ, ). We will then discuss the tradeoff between the robustness to outliers and robustness to adversarial attacks, and will characterize the otimal ψ that achieves this tradeoff. In the location estimator, we will assume ψ(x, θ) is monotonic in θ, which will satisfy the regularity conditions established in [2] Minimizing AIF(ψ, F θ, ). For =, using (5.), we have max ψ (x θ) x AIF(ψ, F θ, ) = E Fθ [ψ (X θ)]. (5.3) For >, using (5.2), we obtain ( [ ψ E Fθ (X θ) AIF(ψ, F θ, ) = E Fθ [ψ (X θ)] ]) In articular, for = 2, we have E Fθ [ψ (X θ) AIF(ψ, F θ, 2) = 2 ] (E Fθ [ψ (X θ)]) 2. From (5.3) and using Jensen s equality, we have AIF(ψ, F θ, ), for which the equality holds when ψ (x θ) is constant in x..

12 2 E. BAYAKTAR AD L. LAI Tradeoff between AIF (ψ, F θ, ) and γ (ψ, F θ ). From (2.3.2) of [3], we know that the influence function of the location estimator secified by ψ is IF(x, ψ, F θ ) = ψ(x θ) E Fθ [ψ (X θ)], and hence γ (ψ, F θ ) = su x ψ(x θ) E Fθ [ψ (X θ)]. As a result, if ψ (X θ) is a constant that minimizes AIF(ψ, F θ, ) as discussed in Section 5.., then γ (ψ, F θ ) might go to, esecially for those distributions with unbounded suort. To achieve a desirable tradeoff between robustness to outliers (i.e., γ (ψ, F θ ) is small) and robustness to adversarial attacks (i.e., AIF(ψ, F θ, ) is small), in the following, we characterize the otimal estimator that minimizes AIF(ψ, F θ, ) subject to a constraint on γ (ψ, F θ ). (5.4) (5.5) (5.6) min AIF(ψ, F, 2) s.t. γ (ψ, F θ ) ξ, E Fθ [ψ(x θ)] = 0, ψ (x) 0, in which constraint (5.5) imlies that γ (ψ, F θ ) is uerbounded by a ositive constant ξ, constraint (5.6) imlies that ψ is Fisher consistent, and the last constraint comes from the condition that ψ is monotonic in θ. For location estimator, f θ (x) = f 0 (x θ), so all quantities in (5.4) remain the same by assuming θ = 0 [3]. Hence, in the following, we will solve this otimization roblem assuming θ = 0. THEOREM 5.. The solution to the otimization roblem (5.4) has the following structure: ψ (x) satisfies (5.7) ψ (x) = { ν ϑ 2 +(ϑ ϑ 2 )F 0(x) f 0 (x), ν f 0 (x) > ϑ F 0(x) + ϑ 2 ( F 0(x)); 0, otherwise, in which the arameters ν, ϑ 0 and ϑ 2 0 are arameters chosen to

13 ADVERSARIAL ROBUSTESS OF ROBUST ESTIMATORS 3 satisfy the following conditions (5.8) (5.9) ϑ ( E F0 [ψ (X)] =, ) ψ (x)f 0 (x)dx ξ = 0, [ X ] ) (5.0) ϑ 2 (E F0 ψ (t)dt ξ = 0, along with [ ] ψ X (x)f 0 (x)dx ξ and E F0 ψ (t)dt ξ. [ ] X ψ() is set as E F0 ψ (t)dt. PROOF. Please see Aendix B for details. The condition ν f 0 (x) > ϑ F 0(x) + ϑ 2 ( F 0(x)) has a natural interretation. It will trim data oints from the tails. In articular, when x is left tail (i.e. F 0 is small), F 0 will be close to. On the other hand, when x is in the right tail (i.e. F 0 is small), F 0 will be close to. In these regions, ψ = 0 if the corresonding f 0 (x) is small. Figure illustrates the scenario for estimating the mean of Gaussian variables for the case assuming ϑ > ϑ 2. It is easy to check that, in this examle, if ν > 2π(ϑ + ϑ 2 ), there exist a and b such that ψ (x) = 0 when x < a or x > b. Corresondingly, ψ(x) is given as ξ ψ(x) = ξ + x a ψ (t)dt ξ x b a < x < b x < a. a b FIG. Gaussian mean examle

14 4 E. BAYAKTAR AD L. LAI 5.2. Scale Estimator. We now secialize the results to the scale model where F θ (x) = F (x/θ). For this model, it is natural to consider ψ(x, θ) = ψ(x/θ) [2, 3]. Similar to the location model, we will first characterize ψ that minimizes AIF(ψ, F θ, ). We will then discuss the tradeoff between the robustness to outliers and robustness to adversarial attacks, and will characterize the otimal ψ that achieves this tradeoff. For the case with =, using (5.), we obtain AIF(ψ, x, ) = T ψ (x n /T ) x n ψ (x n /T ) n= For >, using (5.), we have a.s. max θψ (x/θ) x E θ [Xψ (X/θ)] := AIF(ψ, F θ, ). AIF(ψ, x, ) = ( a.s. n= ) ψ (x n /T ) n= x n /T ψ (x n /T ) ( [ ψ E θ (X/θ) E θ [X/θψ (X/θ)] ]) := AIF(ψ, F θ, ). ( Since in scale model F θ (x) = F (x/θ), we have f θ (x) = f x ) θ θ, and hence AIF(ψ, F θ, ) = For = 2, we have ( [ ψ E F (X) E F [Xψ (X)] ]) := AIF(ψ, F, ). (5.) AIF(ψ, F, 2) = (E F [ψ (X) 2 ]) 2 E F [Xψ (X)] Minimizing AIF (ψ, F θ, ). In the following, among Fisher consistent estimators, we aim to design ψ that minimizes AIF(ψ, F, 2). THEOREM 5.2. structure: The otimal ψ that minimizes AIF(ψ, F, 2) has the following

15 ADVERSARIAL ROBUSTESS OF ROBUST ESTIMATORS 5 For x in the range of f (x), ψ satisfies ψ() is chosen as ψ (x) = x E F [X 2 ]. ψ() = E F [ X ] ψ (t)dt. With this choice of ψ(x), the minimal value of AIF(ψ, F, 2) is / E F [X 2 ]. PROOF. Please see Aendix C for details. We note that for scale estimator ψ θ = ψ (x/θ)x/θ 2, hence for this articular choice of ψ in Theorem 5.2, ψ θ = x2 /(θ 3 E F [X 2 ]), which means ψ(x, θ) is monotone in θ. This ensures that the obtained ψ(x) satisfies the regularity conditions [2] mentioned at the beginning of this section Tradeoff between AIF (ψ, F θ, ) and γ (ψ, F θ ). Similar to the location estimation case, we can also design ψ to minimize AIF(ψ, F θ, ) with a constraint on γ (ψ, F θ ). From (2.3.7) of [3], we know that for scale estimators IF(x, ψ, F θ ) = ψ(x/θ)θ E Fθ [X/θψ (X/θ)]. To facilitate the analysis, we will focus on ψ that is monotonic. Since in scale model, ψ(x, θ) = ψ(x/θ), we can simly focus on the case of θ =. Hence, we will solve the following otimization roblem to strike a desirable tradeoff between robustness against outliers and robustness against adversarial attacks. [ ] E F ψ (X) 2 (5.2) min (E F [Xψ (X)]) 2, (5.3) (5.4) (5.5) s.t. γ (ψ, F ) = su x E F [ψ] = 0, ψ (x) 0. ψ(x) E F [Xψ (X)] ξ, Here, constraint (5.3) is a constraint on the outliers influence, (5.4) imlies that ψ is Fisher consistent. THEOREM 5.3. The solution to (5.2) has the following structure:

16 6 E. BAYAKTAR AD L. LAI ψ has the following form (5.6) ψ (x) = { ν x ϑ 2 +(ϑ ϑ 2 )F (x) f (x), ν xf (x) > ϑ F (x) + ϑ 2 ( F (x)); 0, otherwise, in which ν, ϑ 0 and ϑ 2 0 are chosen to satisfy ϑ ϑ 2 ( E F [Xψ (X)] ) =, = 0, ψ (x)f (x)dx ξ ] (E F [ X ψ (t)dt ) ξ = 0, along with [ ] X ψ (x)f (x)dx ξ and E F ψ (t)dt ξ. [ ] X ψ() is set to be E F ψ (t)dt. PROOF. The roof follows similar strategy as that of the roof of Theorem 5. and 5.2. Details can be found in Aendix D. Similar to the location estimator case, the condition ν xf (x) > ϑ F (x) + ϑ 2 ( F (x)) will limit the influences of data oints at the tails. 6. Extension: L-estimator. In this section, we briefly discuss how to extend the analysis above to other class of estimators. We will L-estimator as an examle. L-estimator has the following form [2, 3]: T (x) = a n x (n), n= where x () x () are the ordered sequence of x, and a n s are coefficients. For examle, for location estimator, a natural choice of a n is (6.) a n = n/ (n )/ h(t)dt 0 h(t)dt, for a given function h(t) such that 0 h(t)dt 0. For examle, setting h(t) = δ(t /2) leads to the median estimator.

17 ADVERSARIAL ROBUSTESS OF ROBUST ESTIMATORS 7 We first look at the given samle scenario. Let x = x + x, and let x () x () be the ordered sequence of x. Hence, (6.2) T (x + x) = a n x (n). n= For general x, the ordering of x + x may not necessarily be the same as the ordering of x. For examle, x () might come from x (2), i.e., x () = x (2) + (x (2) ). This ossibility could make the following analysis messy. However, it is easy to show that for any ˆx that makes the ordering of x+ ˆx different from the ordering of x, we can find another x such that the ordering of x + x is the same as the ordering of x and x ˆx. With this, we can limit (6.2) to the following form Hence and (2.) becomes T (x + x) = a n (x (n) + (x (n) )). n= T (x + x) T (x) = min s.t. a n x (n), n= a n x (n), n= x η. Using the exactly same aroach as those in the roof of Theorem 4., we have the following characterization. For =, let n = arg max a n, n and x (n) = 0, n n. Hence, For >, we have x (n ) = sign {a n } η, AIF(T, x, ) = a n. x (n) = a n /( ) () / ( a n /( ) ) / sign(a n)η.

18 8 E. BAYAKTAR AD L. LAI Hence, (6.3) AIF(ψ, x, ) = a n a n /( ) () / ( a n /( ) ) / sign(a n) = ( a n /( ) n= ) /. a n /( ) n= When = 2, this can be simlified to (6.4) AIF(T, x, 2) = a 2 n n= a 2 n n= = a 2 n. n= For examle, for α-trimmed estimator [3] defined by T α (x) = 2 α α n= α + x (n), for a given arameter 0 < α < /2. For this α-trimmed estimator, using (6.3), we obtain AIF(T, α / x, ) = ( 2 α ) /. If a n s are chosen as (6.), then (6.4) simlifies to AIF(T, x, 2) = n=, ( ( n/ (n )/ h(t)dt ) 2 ( n= ( 0 h(t)dt ) 2 n/ (n )/ h(t)dt ) 2 0 h(t)dt ) 2 in which the first inequality is due to Jensen s inequality, and both inequalities become equality when a n = n/ (n )/ h(t)dt is a constant in n.

19 ADVERSARIAL ROBUSTESS OF ROBUST ESTIMATORS 9 7. umerical Examles. In this section, we rovide numerical examles to illustrate results obtained. We consider location estimation and illustrate the otimal estimator obtained in Theorem 5. for the case when f 0 is exonential random variable f 0 (x) = e x, x 0, hence f θ is shifted exonential random variable f θ = e (x θ), x θ and the goal is to estimate θ. As the exonential random variable has a unbounded suort, choosing ψ to be a constant, which minimizes AIF, will lead to an infinite IF. Hence, we use Theorem 5. to characterize the otimal ψ that minimizes AIF while satisfying the condition that IF ξ. For this articular class of distribution, the condition ν f 0 (x) > ϑ F 0(x) + ϑ 2 ( F 0(x)) becomes 0 x < a with the arameter a chosen as (7.) Hence we have e a = ϑ ν + ϑ. ϑ 2 ψ (x) = { ν + ϑ ϑ 2 ϑ ex, 0 x < a; 0, otherwise, for which the arameters ν, ϑ, ϑ 2 are chosen to satisfy the conditions secified in Theorem 5.. After tedious calculation, conditions (5.8) - (5.0) can be simlified to (ν + ϑ ϑ 2)( e a ) ϑ a =, ϑ ((ν + ϑ ϑ 2)(a + e a ) ϑ (e a ) + aϑ ξ) = 0, ϑ 2((ν + ϑ ϑ 2)( e a ) ϑ a ξ) = 0. From here, we know that if ξ >, ϑ 2 = 0, using this fact along with (7.), we have that the conditions are simlified to ν aϑ =, 2aϑ + (a 2)ν = ξ, ϑ 2 = 0. Using these, we can exress ν and ϑ in terms of a: ν = ξ + 2 a, ϑ = ξ + 2 a a 2.

20 20 E. BAYAKTAR AD L. LAI a FIG 2. The solution of a Finally, for any given ξ >, the value of a can be determined by (7.), which is simlified to (7.2) e a = ϑ ν + ϑ ϑ 2 = ξ + 2 a (ξ + )a + ξ + 2. It is easy to check that, for any given ξ >, there is always a unique ositive solution to (7.2). For examle, Figure 2 illustrates the solution for a when ξ = 3. In this figure, the dotted curve is the right side of (7.2) and the solid curve is the left side of (7.2). From the figure, we know that these two curves have two intersections a = 0 and a = 4.8. With these arameters, we know that (7.3) ψ (x) = { e x, 0 x 4.8; 0, otherwise, hence the otimal ψ is { ψ ξ, x 4.8; (7.4) (x) =.047x (e x ), 0 x 4.8. Figure 3 illustrates the obtained ψ(x) for the case with ξ = 3. Figure 4 illustrates the tradeoff curve between AIF and IF. We obtain this curve by solving (7.2) and other arameters using different values of ξ. As we can see from the curve, as ξ increases, AIF decreases. Furthermore, the value of AIF converges to, the lower bound established in Section Conclusion. Motivated by recent data analytics alications, we have studied adversarial robustness of robust estimators. We have introduced the concet

21 ADVERSARIAL ROBUSTESS OF ROBUST ESTIMATORS (x) x FIG 3. ψ that minimizes AIF when IF AIF IF FIG 4. Tradeoff between AIF and IF of location estimator for exonential random variables. of AIF to quantify an estimator s sensitivity to such adversarial attacks and have rovided an aroach to characterize AIF for given robust estimator. We have further designed otimal estimators that minimize AIF. From this characterization, we have identified a tradeoff between AIF and IF, and have designed estimators that strike a desirable tradeoff between these two quantities. APPEDIX A: PROOF OF THEOREM 4. From (3.), we know that T and x satisfy ψ(x n, T ) = 0. n=

22 22 E. BAYAKTAR AD L. LAI Hence, we have (A.) x n T = x [ψ] x=x n,θ=t. θ [ψ] x=x n,θ=t n= Based on Taylor exansion, we have T (x + x) T (x) = x n T + higher order terms. x n n= When η is small, the adversary can solve the following roblem and obtain an o(η) otimal solution (A.2) in which min x x n c n, n= s.t. x η, c n := x n T. For =, this is a linear rograming roblem, whose solution is simle. In articular, let n = arg max n x n T, which is the same as arg max n x [ψ] x=x n,θ=t due to (A.), it is easy to check that we have { } x n = sign T η, x n and x n = 0, n n. Hence, AIF(ψ, x, ) = T = x n x [ψ] x=x n,θ=t. θ [ψ] x=x n,θ=t n= For >, (A.2) is a convex otimization roblem. To solve this, we form Lagrange L( x, λ) = x n c n + λ ( x η ). n=

23 ADVERSARIAL ROBUSTESS OF ROBUST ESTIMATORS 23 The corresonding otimality conditions are: (A.3) c n + λ sign( x n) x n = 0, n λ 0, λ ( x η ) = 0. From (A.3), we know that λ 0, hence (A.4) x = η, and (A.5) sign( x n) x n = c n λ. From (A.5) and the fact that λ is ositive, we know sign( x n) = sign(c n ), and hence we have which can be simlified further to x n = x n = c n λ, ( ) cn /( ) λ sign(c n ). Combining these with (A.4), we obtain the value of λ : λ = c n /( ) n= η ( )/. As the result, we have Hence, x n = c n /( ) () / ( c n /( ) ) / sign(c n)η. AIF(ψ, x, ) = c n c n /( ) () / ( c n /( ) ) / sign(c n) = c n /( ) n= ( cn /( )) /.

24 24 E. BAYAKTAR AD L. LAI Using (A.), we can further simlify the exression to AIF(ψ, x, ) = ( x n [ψ] x=xn,θ=t. θ [ψ] x=x n,θ=t n= n= ) APPEDIX B: PROOF OF THEOREM 5. As ψ (x) 0, we have E F0 [ψ (X)] > 0, and su ψ(x) is either ψ( ) or ψ(). Hence for = 2, the otimization roblem (5.4) is equivalent to x min s.t. E F0 [ψ (X) 2 ] (E F0 [ψ (X)]) 2 ψ() + ψ (x)dx ξ, E F0 [ψ (X)] ψ() E F0 [ψ (X)] ξ, [ X ] ψ() + E F0 ψ (t)dt = 0, ψ 0. As the objective function does not involve ψ(), we can first solve min s.t. E F0 [ψ (X) 2 ] (E F0 [ψ (X)]) 2, [ ] X E F0 ψ (t)dt E F0 [ψ (X)] [ ] X E F0 ψ (t)dt ξ, E F0 [ψ (X)] + ψ (x)dx ξ, ψ (x) 0. [ ] X After obtaining the solution, we can simly set ψ() = E F0 ψ (t)dt to make ψ Fisher consistent. To simlify the notation, in the remainder of the roof, we will use g(x) to

25 ADVERSARIAL ROBUSTESS OF ROBUST ESTIMATORS 25 denote ψ (x). We now further simlify the otimization roblem. First, we have (B.) E F0 [ X ] g(t)dt = = = [ x f 0 (x) [ g(t) t ] g(t)dt dx ] f 0 (x)dx dt g(t) [ F 0 (t)] dt. Couled with the fact that g(x) 0 and f 0 (x) 0, the otimization above is equivalent to min s.t. g2 (x)f 0 (x)dx ( ) 2, g(x)f 0(x)dx g(x) 0. g(x)f 0 (x)dx ξ g(x) [ F 0 (x)] dx ξ g(x)f 0 (x)dx, g(x)f 0 (x)dx, It is clear that the otimization roblem is scale invariant in the sense that if g (x) is a solution to this roblem, then for any ositive constant c, cg (x) is also a solution to this roblem. As a result, without loss of generality, we can assume g(x)f 0(x)dx =. Using this, we can further simlify the otimization roblem to min s.t. 2 g(x) 0. g 2 (x)f 0 (x)dx, g(x)f 0 (x)dx =, g(x)f 0 (x)dx ξ, g(x) [ F 0 (x)] dx ξ, To solve this functional minimization roblem, we first form the Lagrangian

26 26 E. BAYAKTAR AD L. LAI function L = ( g 2 (x)f 0 (x)dx + ν 2 ( ) ( +ϑ g(x)f 0 (x)dx ξ + ϑ 2 ) g(x)f 0 (x)dx + λ(x)g(x) ). g(x) [ F 0 (x)] dx ξ Let H = 2 g2 (x)f 0 (x) νg(x)f 0 (x) + ϑ g(x)f 0 (x) + ϑ 2 g(x)( F 0 (x)) λ(x)g(x). As no derivative g (x) is involved in H, the otimality condition Euler- Lagrange equation [9] simlifies to H g d dx ( ) H = 0 g (B.2) g (x)f 0 (x) ν f 0 (x) + ϑ 2 + (ϑ ϑ 2)F 0 (x) λ (x) = 0, in which the arameters ϑ 0, ϑ 2 0, λ (x) 0 satisfy [8] (B.3) g (x)f 0 (x)dx =, ( ) ϑ g (x)f 0 (x)dx ξ = 0, ( ) ϑ 2 g (x) [ F 0 (x)] dx ξ = 0, λ (x)g(x) 0. From (B.2), for x in the range of f 0 (x), we have g (x) = λ (x) + ν f 0 (x) ϑ 2 (ϑ ϑ 2 )F 0(x). f 0 (x) Combining this with the condition (B.3), we know that if ν f 0 (x) ϑ 2 (ϑ ϑ 2 )F 0(x) > 0, then λ (x) = 0. On the other hand, if ν f 0 (x) ϑ 2 (ϑ ϑ 2 )F 0(x) < 0, then g (x) = 0. As a result, we have g (x) = { ν ϑ 2 +(ϑ ϑ 2 )F 0(x) f 0 (x), ν f 0 (x) > ϑ F 0(x) + ϑ 2 ( F 0(x)); 0, otherwise, which comletes the roof.

27 ADVERSARIAL ROBUSTESS OF ROBUST ESTIMATORS 27 APPEDIX C: PROOF OF THEOREM 5.2 First of all, minimizing (5.) is same as solving [ ] E F ψ (X) 2 (C.) min (E F [Xψ (X)]) 2, (C.2) s.t. E F [ψ(x)] = ψ() + E F [ X ] ψ (t)dt = 0, in which the condition E F [ψ(x)] = 0 ensures that the estimator is Fisher consistent. As ψ() does not aear in the objective function, we can solve (C.) without the constraint (C.2) first. After that, we can simly set ψ() = E F [ X ] ψ (t)dt so that the constraint (C.2) will be satisfied. Furthermore, similar to the roof of Theorem 5., to simlify the notation, we will use g(x) to denote ψ (x). It is clear from (C.) that the cost function is scale-invariant. Hence, without loss of generality, we can assume (E F [Xg(X)]) 2 =, for which we can further focus on E F [Xg(X)] =. Combining all these together, the otimization roblem can be converted to min s.t. 2 g 2 (x)f (x)dx, xg(x)f (x)dx =. For this calculus of variations roblem, we form Lagrange function ( ) L = 2 g2 (x)f (x)dx + ν xg(x)f (x)dx. The corresonding Euler-Lagrange equation can be simlified to (C.3) g (x)f (x) ν xf (x) = 0, and the otimal value of ν is selected to satisfy the condition (C.4) xg (x)f (x)dx =.

28 28 E. BAYAKTAR AD L. LAI From (C.3), we know that in the range of X where f (x) > 0, g (x) = ν x. Plugging this into (C.4), we obtain ν = x2 f (x)dx. As the result, for x in the range of f (x), the otimal g (x) is g x (x) = E F [X 2 ], and ψ() = E F [ X g(t)dt ]. APPEDIX D: PROOF OF THEOREM 5.3 Following the same strategy as those in the roof of Theorem 5., we can first solve the following roblem [ ] E F ψ (X) 2 min (E F [Xψ (X)]) 2, [ X ] s.t. E F ψ (t)dt + ψ (t)dt ξ E F [Xψ (X)], E F [ X ] ψ (t)dt ξ E F [Xψ (X)], ψ (x) 0, [ ] X and then set ψ() = E F ψ (t)dt to satisfy the Fisher consistent constraint (5.4). ow, we consider two different cases deending on whether E F [Xψ (X)] is ositive or negative. In the following, to simlify notation, we will use g(x) to denote ψ (x). We will solve the case with E F [Xg(X)] > 0 in detail. The case E F [Xg(X)] < 0 can be solved in the similar manner. With E F [Xg(X)] > 0, the otimization roblem is same as min s.t. E F [ g 2 (X) ] (E F [Xg(X)]) 2, [ X ] E F g(t)dt E F [ X g(x) 0. + g(t)dt ξe F [Xg(X)], ] g(t)dt ξe F [Xg(X)],

29 ADVERSARIAL ROBUSTESS OF ROBUST ESTIMATORS 29 Similar to the otimization roblems in Theorem 5. and 5.2, the otimization roblem is scale-invariant, and hence without loss of generality, [ we can focus on ] X E F [Xg(X)] =. Furthermore, similar to (B.), we have E F g(t)dt = g(t) [ F (t)] dt. The roblem is then converted to min s.t. g(x) 0. g 2 (x)f (x)dx, xg(x)f (x)dx =, g(x)f (x)dx ξ, g(x)[ F (x)]dx ξ, To solve this functional minimization roblem, we first form the Lagrangian function L = ( ) g 2 (x)f (x)dx + ν xg(x)f (x)dx + λ(x)g(x) 2 ( ) ( ) +ϑ g(x)f (x)dx ξ + ϑ 2 g(x) [ F (x)] dx ξ. Let F = 2 g2 (x)f (x) νxg(x)f (x) + ϑ g(x)f (x) + ϑ 2 g(x)( F (x)) λ(x)g(x). As no derivative g (x) is involved in F, the Euler-Lagrange equation F g d ( ) F = 0 dx g simlifies to (D.) g (x)f (x) ν xf (x) + ϑ 2 + (ϑ ϑ 2)F (x) λ (x) = 0, in which the arameters ϑ 0, ϑ 2 0, λ (x) 0 satisfy (D.2) xg (x)f (x)dx =, ( ) ϑ g (x)f (x)dx ξ = 0, ( ) ϑ 2 g (x) [ F (x)] dx ξ = 0, λ (x)g (x) 0.

30 30 E. BAYAKTAR AD L. LAI From (D.), for those x with f (x) > 0, we have g (x) = λ (x) + ν xf (x) ϑ 2 (ϑ ϑ 2 )F (x). f (x) Combining this with the condition (D.2), we know that if ν xf (x) ϑ 2 (ϑ ϑ 2 )F (x) > 0, then λ (x) = 0. On the other hand, if ν f (x) ϑ 2 (ϑ ϑ 2 )F (x) < 0, then g (x) = 0. As the result, we have g (x) = { ν x ϑ 2 +(ϑ ϑ 2 )F (x) f (x), ν xf (x) > ϑ F (x) + ϑ 2 ( F (x)); 0, otherwise. REFERECES [] Huber, P. J. (964). Robust estimation of a location arameter. Ann. Math. Statist., [2] Huber, P. J. and Ronchetti, E. (2009). Robust statistics. Wiley, ew York. [3] Hamel, F., Ronchetti, E., Rousseeuw, P., and Stahel, W. (2009). Robust statistics: The Aroach Based on Influence Functions. Wiley, Hoboken, J. [4] Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I. J. and Fergus, R. (203). Intriguing roerties of neural networks. CoRR, vol. abs/ [5] Goodfellow, I. J., Shlens, J. and Szegedy, C. (205). Exlaining and harnessing adversarial examles. in Proc. International Conference on Learning Reresentations, San Diego, CA. [6] Carlini,. and Wagner, D. (207). Towards evaluating the robustness of neural networks. In Proc. IEEE Intl. Symosium on Security and Privacy, San Jose, CA. [7] Hamel, F. (968). Contributions to the theory of robust estimation. Ph.D. thesis, University of California, Berkeley. [8] Burger, M. (2003). Infinite-dimensional Otimization and Otimal Design. Lecture notes, available at ft://ft.math.ucla.edu/ub/camreort/cam04-.df. [9] Kot, M. (204). A first course in the calculus of variations. Providence, Rhode Island: American Mathematical Society. [0] Pimentel-Alarcon, D. L., Biswas, A. and Solis-Lemus, C. R. (207). Adversarial rincial comonent analysis. In Proc. IEEE Intl. Symosium on Inform. Theory, [] Jagielski, M., Orea, A., Biggio, B., Liu, C., ita-rotaru, C. and Li, B. (208). Maniulating machine learning: Poisoning attacks and countermeasures for regression learning. In Proc. IEEE Intl. Symosium on Security and Privacy, San Francisco, CA. [2] Biggio, B., elson, B. and Laskov, P. (202). Poisoning attacks against suort vector machines. In Proc. International Conference on Machine Learning, Edinburgh, Scotland. [3] Charikar, M., Steinhardt, J., and Valiant, G. (207). Learning from untrusted data. In Proc. Annual ACM SIGACT Symosium on Theory of Comuting. [4] Croux, C. (998). Limit behavior of the emirical influence function of the median. Statistics & Probability Letter, DEPARTMET OF MATHEMATICS UIVERSITY OF MICHIGA A ARBOR, MI erhan@umich.edu DEPARTMET OF ELECTRICAL AD COMPUTER EGIEERIG UIVERSITY OF CALIFORIA DAVIS, CA, lflai@ucdavis.edu