A theory of learning from different domains

Transcription

1 DOI /s A theory of learning fro different doains Shai Ben-David John Blitzer Koby Craer Alex Kulesza Fernando Pereira Jennifer Wortan Vaughan Received: 28 February 2009 / Revised: 12 Septeber 2009 / Accepted: 18 Septeber 2009 The Authors This article is published with open access at Springerlink.co Abstract Discriinative learning ethods for classification perfor well when training and test data are drawn fro the sae distribution. Often, however, we have plentiful labeled training data fro a source doain but wish to learn a classifier which perfors well on a target doain with a different distribution and little or no labeled training data. In this work we investigate two questions. First, under what conditions can a classifier trained fro source data be expected to perfor well on target data? Second, given a sall aount of labeled target data, how should we cobine it during training with the large aount of labeled source data to achieve the lowest target error at test tie? Editors: Nicolo Cesa-Bianchi, David R. Hardoon, and Gayle Leen. Preliinary versions of the work contained in this article appeared in Advances in Neural Inforation Processing Systes Ben-David et al. 2006; Blitzer et al. 2007a. S. Ben-David David R. Cheriton School of Coputer Science, University of Waterloo, Waterloo, ON, Canada e-ail: shai@cs.uwaterloo.ca J. Blitzer Departent of Coputer Science, UC Berkeley, Berkeley, CA, USA e-ail: blitzer@cs.berkeley.edu K. Craer Departent of Electrical Engineering, The Technion, Haifa, Israel e-ail: koby@ee.technion.ac.il A. Kulesza Departent of Coputer and Inforation Science, University of Pennsylvania, Philadelphia, PA, USA e-ail: kulesza@cis.upenn.edu F. Pereira Google Research, Mountain View, CA, USA e-ail: pereira@google.co J.W. Vaughan School of Engineering and Applied Sciences, Harvard University, Cabridge, MA, USA e-ail: jenn@seas.harvard.edu

2 We address the first question by bounding a classifier s target error in ters of its source error and the divergence between the two doains. We give a classifier-induced divergence easure that can be estiated fro finite, unlabeled saples fro the doains. Under the assuption that there exists soe hypothesis that perfors well in both doains, we show that this quantity together with the epirical source error characterize the target error of a source-trained classifier. We answer the second question by bounding the target error of a odel which iniizes a convex cobination of the epirical source and target errors. Previous theoretical work has considered iniizing just the source error, just the target error, or weighting instances fro the two doains equally. We show how to choose the optial cobination of source and target error as a function of the divergence, the saple sizes of both doains, and the coplexity of the hypothesis class. The resulting bound generalizes the previously studied cases and is always at least as tight as a bound which considers iniizing only the target error or an equal weighting of source and target errors. Keywords Doain adaptation Transfer learning Learning theory Saple-selection bias 1 Introduction Most research in achine learning, both theoretical and epirical, assues that odels are trained and tested using data drawn fro soe fixed distribution. This single doain setting has been well studied, and unifor convergence theory guarantees that a odel s epirical training error is close to its true error under such assuptions. In any practical cases, however, we wish to train a odel in one or ore source doains and then apply it to a different target doain. For exaple, we ight have a spa filter trained fro a large eail collection received by a group of current users the source doain and wish to adapt it for a new user the target doain. Intuitively this should iprove filtering perforance for the new user, under the assuption that users generally agree on what is spa and what is not. The challenge is that each user receives a unique distribution of eail. Many other exaples arise in natural language processing. In general, labeled data for tasks like part-of-speech tagging Ratnaparkhi 1996, parsing Collins 1999, inforation extraction Bikel et al. 1997, and sentient analysis Pang et al are drawn fro a liited set of docuent types and genres in a given language due to availability, cost, and specific goals of the project. However, useful applications for the trained systes ay involve docuents of different types or genres. We can hope to successfully adapt the systes in these cases since parts-of-speech, syntactic structure, entity entions, and positive or negative sentient are to a large extent stable across different doains, as they depend on general properties of language. In this work we investigate the proble of doain adaptation. We analyze a setting in which we have plentiful labeled training data drawn fro one or ore source distributions but little or no labeled training data drawn fro the target distribution of interest. This work answers two ain questions. First, under what conditions on the source and target distributions can we expect to learn well? We give a bound on a classifier s target doain error in ters of its source doain error and a divergence easure between the two doains. In a distribution-free setting, we cannot obtain accurate estiates of coon easures of divergence such as L 1 or Kullback-Leibler fro finite saples. Instead, we show that when learning a hypothesis fro a class of finite coplexity, it is sufficient to use a classifier-induced divergence we call the H H-divergence Kifer et al. 2004;

3 Ben-David et al Finite saple estiates of the H H-divergence converge uniforly to the true H H-divergence, allowing us to estiate the doain divergence fro unlabeled data in both doains. Our final bound on the target error is in ters of the epirical source error, the epirical H H-divergence between unlabeled saples fro the doains, and the cobined error of the best single hypothesis for both doains. A second iportant question is how to learn when the large quantity of labeled source data is augented with a sall aount of labeled target data, for exaple, when our new eail user has begun to anually ark a few received essages as spa. Given a source doain S and a target doain T, we consider hypotheses h which iniize a convex cobination of epirical source and target error ˆɛ T h and ˆɛ S h, respectively, which we refer to as the epirical α-error: α ˆɛ T h + 1 αˆɛ S h. Setting α involves trading off the ideal but sall target dataset against the large but less relevant source dataset. Baseline choices for α include α = 0 using only source data Ben- David et al. 2006, α = 1 using only target data, and the equal weighting of source and target instances Craer et al. 2008, setting α to the fraction of the instances that are fro the target doain. We give a bound on a classifier s target error in ters of its epirical α error. The α that iniizes the bound depends on the divergence between the doains as well as the size of the source and target training datasets. The optial bound is always at least as tight as the bounds using only source, only target, or equally-weighted source and target instances. We show that for a real-world proble of sentient classification, nontrivial settings of α perfor better than the three baseline settings. In the next section, we give a brief overview of related work. We then specify precisely our odel of doain adaptation. Section 4 shows how to bound the target error of a hypothesis in ters of its source error and the source-target divergence. Section 5 gives our ain result, a bound on the target error of a classifier which iniizes a convex cobination of epirical errors on the two doains, and in Sect. 6 we investigate the properties of the best convex cobination of that bound. In Sect. 7, we illustrate experientally the above bounds on sentient classification data. Section 8 describes how to extend the previous results to the case of ultiple data sources. Finally, we conclude with a brief discussion of future directions for research in Sect Related work Craer et al introduced a PAC-style odel of learning fro ultiple sources in which the distribution over input points is assued to be the sae across sources but each source ay have its own deterinistic labeling function. They derive bounds on the target error of the function that iniizes the epirical error on uniforly weighted data fro any subset of the sources. As discussed in Sect. 8.2, the bounds that they derive are equivalent to ours in certain restricted settings, but their theory is significantly less general. Daué 2007 and Finkel 2009 suggest an epirically successful ethod for doain adaptation based on ulti-task learning. The crucial difference between our doain adaptation setting and analyses of ulti-task ethods is that ulti-task bounds require labeled data fro each task, and ake no attept to exploit unlabeled data. Although these bounds have a ore liited scope than ours, they can soeties yield useful results even when the optial predictors for each task or doain in the case of Daué 2007 are quite different Baxter 2000; Ando and Zhang 2005.

4 Li and Biles 2007 give PAC-Bayesian learning bounds for adaptation using divergence priors. In particular, they place a source-centered prior on the paraeters of a odel learned in the target doain. Like our odel, the divergence prior ephasizes the tradeoff between source hypotheses trained on large but biased data sets and target hypotheses trained fro sall but unbiased data sets. In our odel, however, we easure the divergence and consequently the bias of the source doain fro unlabeled data. This allows us to choose a tradeoff paraeter for source and target labeled data before training begins. More recently, Mansour et al. 2009a, 2009b introduced a theoretical odel for the ultiple source adaptation proble. This odel operates under assuptions very siilar to our ultiple source analysis Sect. 8, and we address their work in ore detail there. Finally, doain adaptation is closely related to the setting of saple selection bias Heckan A well-studied variant of this is covariate shift, which has seen significant work in recent years Huang et al. 2007; Sugiyaa et al. 2008; Cortesetal This line of work leads to algoriths based on instance weighting, which have also been explored epirically in the achine learning and natural language processing counities Jiang and Zhai 2007;Bickeletal Our work differs fro covariate shift priarily in two ways. First, we do not assue the labeling rule is identical for the source and target data although there ust exist soe good labeling rule for both in order to achieve low error. Second, our H H-divergence can be coputed fro finite saples of unlabeled data, allowing us to directly estiate the error of a source-trained classifier on the target doain. A point of general contrast is that we work in an agnostic setting in which we do not ake strong assuptions about the data generation odel, such as a specific relationship between the source and target data distributions, which would be needed to obtain absolute error bounds. Instead, we assue only that the saples fro each of the two doains are generated i.i.d. according to the respective data distributions, and as a result our bounds ust be relative to the error of soe benchark predictor rather than absolute, specifically, relative to the cobined error on both doains of an optial joint predictor. 3 A rigorous odel of doain adaptation We foralize the proble of doain adaptation for binary classification as follows. We define a doain 1 as a pair consisting of a distribution D on inputs X and a labeling function f : X [0, 1], which can have a fractional expected value when labeling occurs nondeterinistically. Initially, we consider two doains, a source doain and a target doain. We denote by D S,f S the source doain and D T,f T the target doain. A hypothesis is a function h : X {0, 1}. The probability according to the distribution D S that a hypothesis h disagrees with a labeling function f which can also be a hypothesis is defined as ɛ S h, f = E x DS [ hx fx ]. When we want to refer to the source error soeties called risk of a hypothesis, we use the shorthand ɛ S h = ɛ S h, f S. We write the epirical source error as ˆɛ S h. Weusethe parallel notation ɛ T h, f, ɛ T h,andˆɛ T h for the target doain. 1 Note that this notion of doain is not the doain of a function. We always ean a specific distribution and function pair when we say doain.

5 4 A bound relating the source and target error We now proceed to develop bounds on the target doain generalization perforance of a classifier trained in the source doain. We first show how to bound the target error in ters of the source error, the difference between labeling functions f S and f T, and the divergence between the distributions D S and D T. Since we expect the labeling function difference to be sall in practice, we focus here on easuring distribution divergence, and especially on how to estiate it with finite saples of unlabeled data fro D S and D T. That is the role of the H-divergence introduced in Sect A natural easure of divergence for distributions is the L 1 or variation divergence d 1 D, D = 2sup Pr D [B] Pr D [B], B B where B is the set of easurable subsets under D and D. We ake use of this easure to state an initial bound on the target error of a classifier. Theore 1 For a hypothesis h, ɛ T h ɛ S h + d 1 D S, D T + in { [ E DS fs x f T x ] [, E DT fs x f T x ]}. Proof See Appendix. In this bound, the first ter is the source error, which a training algorith ight seek to iniize, and the third is the difference in labeling functions across the two doains, which we expect to be sall. The proble is the reaining ter. Bounding the error in ters of the L 1 divergence between distributions has two disadvantages. First, it cannot be accurately estiated fro finite saples of arbitrary distributions Batu et al. 2000; Kifer et al and therefore has liited usefulness in practice. Second, for our purposes the L 1 divergence is an overly strict easure that unnecessarily inflates the bound, since it involves a supreu over all easurable subsets. We are only interested in the error of a hypothesis fro soe class of finite coplexity, thus we can restrict our attentions to the subsets on which this type of hypothesis can coit errors. The divergence easure introduced in the next section addresses both of these concerns. 4.1 The H-divergence Definition 1 Based on Kifer et al Given a doain X with D and D probability distributions over X,letH be a hypothesis class on X and denote by Ihthe set for which h H is the characteristic function; that is, x Ih hx = 1. The H-divergence between D and D is d H D, D = 2sup Pr D [Ih] Pr D [Ih]. h H The H-divergence resolves both probles associated with the L 1 divergence. First, for hypothesis classes H of finite VC diension, the H-divergence can be estiated fro finite saples see Lea 1 below. Second, the H-divergence for any class H is never larger than the L 1 divergence, and is in general saller when H has finite VC diension. Since it plays an iportant role in the rest of this work, we now state a slight odification of Theore 3.4 of Kifer et al as a lea.

6 Lea 1 Let H be a hypothesis space on X with VC diension d. If U and U are saples of size fro D and D respectively and ˆd H U, U is the epirical H-divergence between saples, then for any δ 0, 1, with probability at least 1 δ, d H D, D ˆd H U, U d log2 + log 2 δ + 4. Lea 1 shows that the epirical H-divergence between two saples fro distributions D and D converges uniforly to the true H-divergence for hypothesis classes H of finite VC diension. The next lea shows that we can copute the H-divergence by finding a classifier which attepts to separate one doain fro the other. Our basic plan of attack will be as follows: Label each unlabeled source instance with 0 and unlabeled target instance as 1. Then train a classifier to discriinate between source and target instances. The H-divergence is iediately coputable fro the error. Lea 2 For a syetric hypothesis class H one where for every h H, the inverse hypothesis 1 h is also in H and saples U, U of size [ 1 ˆd H U, U = 2 1 in I [x U] + 1 I [ x U ]], h H x:hx=0 x:hx=1 where I[x U] is the binary indicator variable which is 1 when x U. Proof See Appendix. This lea leads directly to a procedure for coputing the H-divergence. We first find a hypothesis in H which has iniu error for the binary classification proble of distinguishing source fro target instances. The error of this hypothesis is related to the H- divergence by Lea 2. Of course, iniizing error for ost reasonable hypothesis classes is a coputationally intractable proble. Nonetheless, as we shall see in Sect. 7, the error of hypotheses trained to iniize convex upper bounds on error are useful in approxiating the H-divergence. 4.2 Bounding the difference in error using the H-divergence The H-divergence allows us to estiate divergence fro unlabeled data, but in order to use it in a bound we ust have tools to represent error relative to other hypotheses in our class. We introduce two new definitions. Definition 2 The ideal joint hypothesis is the hypothesis which iniizes the cobined error h =argin ɛ S h + ɛ T h. h H We denote the cobined error of the ideal hypothesis by λ = ɛ S h + ɛ T h.

7 The ideal joint hypothesis explicitly ebodies our notion of adaptability. When this hypothesis perfors poorly, we cannot expect to learn a good target classifier by iniizing source error. On the other hand, we will show that if the ideal joint hypothesis perfors well, we can easure adaptability of a source-trained classifier by using the H-divergence between the arginal distributions D S and D T. Next we define the syetric difference hypothesis space H H for a hypothesis space H, which will be very useful in reasoning about error. Definition 3 For a hypothesis space H,thesyetric difference hypothesis space H H is the set of hypotheses g H H gx = hx h x for soe h, h H, where is the XOR function. In words, every hypothesis g H H is the set of disagreeents between two hypotheses in H. The following siple lea shows how we can ake use of the H H-divergence in bounding the error of our hypothesis. Lea 3 For any hypotheses h, h H, ɛ S h, h ɛ T h, h 1 2 d H HD S, D T. Proof By the definition of H H-distance, [ d H H D S, D T = 2 sup Prx DS hx h x ] [ Pr x DT hx h x ] h,h H = 2 sup ɛ S h, h ɛ T h, h 2 ɛ S h, h ɛ T h, h. h,h H We are now ready to give a bound on target error in ters of the new divergence easure we have defined. Theore 2 Let H be a hypothesis space of VC diension d. If U S, U T are unlabeled saples of size each, drawn fro D S and D T respectively, then for any δ 0, 1, with probability at least 1 δ over the choice of the saples, for every h H: ɛ T h ɛ S h ˆd H H U S, U T + 4 2d log2 + log 2 δ + λ. Proof This proof relies on Lea 3 and the triangle inequality for classification error Ben- David et al. 2006; Craeretal.2008, which iplies that for any labeling functions f 1, f 2,andf 3,wehaveɛf 1,f 2 ɛf 1,f 3 + ɛf 2,f 3.Then ɛ T h ɛ T h + ɛ T h, h ɛ T h + ɛ S h, h + ɛ T h, h ɛ S h, h ɛ T h + ɛ S h, h d H HD S, D T

8 ɛ T h + ɛ S h + ɛ S h d H HD S, D T = ɛ S h d H HD S, D T + λ ɛ S h ˆd H H U S, U T + 4 2d log2 + log 2 δ + λ. The last step is an application of Lea 1, together with the observation that since we can represent every g H H as a linear threshold network of depth 2 with 2 hidden units, the VC diension of H H is at ost twice the VC diension of H Anthony and Bartlett The bound in Theore 2 is relative to λ, and we briefly coent that the for λ = ɛ S h + ɛ T h coes fro the use of the triangle inequality for classification error. Other losses result in other fors for this bound Craer et al When the cobined error of the ideal joint hypothesis is large, then there is no classifier that perfors well on both the source and target doains, so we cannot hope to find a good target hypothesis by training only on the source doain. On the other hand, for sall λ the ost relevant case for doain adaptation, the bound shows that source error and unlabeled H H-divergence are iportant quantities in coputing the target error. 5 A learning bound cobining source and target training data Theore 2 shows how to relate source and target error. We now proceed to give a learning bound for epirical risk iniization using cobined source and target training data. At train tie a learner receives a saple S = S T,S S of instances, where S T consists of instances drawn independently fro D T and S S consists of 1 instances drawn independently fro D S. The goal of a learner is to find a hypothesis that iniizes target error ɛ T h. When is sall, as in doain adaptation, iniizing epirical target error ay not be the best choice. We analyze learners that instead iniize a convex cobination of epirical source and target error, ˆɛ α h = α ˆɛ T h + 1 αˆɛ S h, for soe α [0, 1]. Wedenoteasɛ α h the corresponding weighted cobination of true source and target errors, easured with respect to D S and D T. We bound the target error of a doain adaptation algorith that iniizes ˆɛ α h. The proof of the bound has two ain coponents, which we state as leas below. First we bound the difference between the target error ɛ T h and weighted error ɛ α h. Thenwe bound the difference between the true and epirical weighted errors ɛ α h and ˆɛ α h. Lea 4 Let h be a hypothesis in class H. Then 1 ɛ α h ɛ T h 1 α 2 d H HD S, D T + λ. Proof See Appendix.

9 The lea shows that as α approaches 1, we rely increasingly on the target data, and the distance between doains atters less and less. The unifor convergence bound on the α-error is nearly identical to the standard unifor convergence bound for hypothesis classes of finite VC diension Vapnik 1998; Anthony and Bartlett 1999, only with target and source errors weighted differently. The key part of the proof relies on a slight odification of Hoeffding s inequality for our setup, which we state here: Lea 5 For a fixed hypothesis h, if a rando labeled saple of size is generated by drawing points fro D T and 1 points fro D S, and labeling the according to f S and f T respectively, then for any δ 0, 1, with probability at least 1 δ over the choice of the saples, Pr [ ˆɛ α h ɛ α h ɛ ] 2ɛ 2 2exp α α2 1 Before giving the proof, we first restate Hoeffding s inequality for copleteness. Proposition 1 Hoeffding s inequality If X 1,...,X n are independent rando variables with a i X i b i for all i, then for any ɛ>0, where X = X 1 + +X n /n. We are now ready to prove the lea. Pr [ X E[ X] ɛ ] 2e 2n2 ɛ 2 / n i=1 b i a i 2, Proof Lea 5 LetX 1,...,X be rando variables that take on the values α hx f T x for the instances x S T. Siilarly, let X +1,...,X be rando variables that take on the values 1 α 1 hx f Sx for the 1 instances x S S. Note that X 1,...,X [0,α/] and X +1,...,X [0,1 α/1 ].Then ˆɛ α h = α ˆɛ T h + 1 αˆɛ S h = α 1 1 hx f T x +1 α hx f S x = 1 1 x S T x S S Furtherore, by linearity of expectations E[ˆɛ α h]= 1 α ɛ T h α 1 ɛ Sh = αɛ T h + 1 αɛ S h = ɛ α h.. X i. i=1

10 So by Hoeffding s inequality the following holds for every h. Pr [ ˆɛ α h ɛ α h ɛ ] 2 2 ɛ 2 2exp i=1 range2 X i 2 2 ɛ 2 = 2exp α α 1 2 2ɛ 2 = 2exp α α2 1. This lea shows that as α oves away fro where each instance is weighted equally, our finite saple approxiation to ɛ α h becoes less reliable. We can now ove on to the ain theore of this section. Theore 3 Let H be a hypothesis space of VC diension d. Let U S and U T be unlabeled saples of size each, drawn fro D S and D T respectively. Let S be a labeled saple of size generated by drawing points fro D T and 1 points fro D S and labeling the according to f S and f T, respectively. If ĥ H is the epirical iniizer of ˆɛ α h on S and h T = in h H ɛ T h is the target error iniizer, then for any δ 0, 1, with probability at least 1 δ over the choice of the saples, ɛ T ĥ ɛ T h T + 4 α α α 2 ˆd H H U S, U T + 4 2d log log 8 δ 2d log2 + log 8 δ + λ. The proof follows the standard set of steps for proving learning bounds Anthony and Bartlett 1999, using Lea 4 to bound the difference between target and weighted errors and Lea 5 for the unifor convergence of epirical and true weighted errors. The full proof is in Appendix. When α = 0 that is, we ignore target data, the bound is identical to that of Theore 2, but with an epirical estiate for the source error. Siilarly when α = 1 that is, we use only target data, the bound is the standard learning bound using only target data. At the optial α which iniizes the right hand side, the bound is always at least as tight as either of these two settings. Finally note that by choosing different values of α, the bound allows us to effectively trade off the sall aount of target data against the large aount of less relevant source data. We reark that when it is known that λ = 0, the dependence on in Theore 3 can be iproved; this corresponds to the restricted or realizable setting. 6 Optial ixing value We exaine now the bound of Theore 3 in ore detail to illustrate soe interesting properties. Writing the bound as a function of α and oitting additive constants, we obtain α 2 1 α2 fα= 2B αa, 1 1

11 where 1 A = 2 ˆd 2d log2 + log 4 δ H H U S, U T λ, is the total divergence between source and target, and 2d log log 8 δ B = 4 is the coplexity ter, which is approxiately d/. The optial value α is a function of the nuber of target exaples T =, the nuber of source exaples S = 1, and the ratio D = d/a: { 1 α T D T, S ; D = 2 in{1,ν} T D 2, 2 where T ν = T + S 1 + S D2 S + T S T. Several observations follow fro this analysis. First, if T = 0 = 0 then α = 0and if S = 0 = 1 then α = 1. That is, if we have only source or only target data, the best cobination is to use exactly what we have. Second, if we are certain that the source and target are the sae, that is if A = 0orD, then α =, that is, the optial cobination is to use the training data with unifor weighting of the exaples across all exaples, as in Craer et al. 2008, who always enforce such a unifor weighing. Finally, two phase transitions occur in the value of α. First, if there are enough target data specifically, if T D 2 = d/a 2 then no source data are needed, and in fact using any source data will yield suboptial perforance. This is because the possible reduction in error due to additional source data is always less than the increase in error caused by the source data being too far fro the target data. Second, even if there are few target exaples, it ight be the case that we do not have enough source data to justify using it, and this sall aount of source data should be ignored. Once we have enough source data then we get a non-trivial value for α. These two phase transitions are illustrated in Fig. 1. The intensity of a point reflects the value α and ranges fro 0 white to 1 black. In this plot α is a function of S x axis and T y axis, and we fix the coplexity to d = 1,601 and the divergence between source and target to A = We chose these values to correspond ore closely to real data see Sect. 7. Observe first that D 2 = 1,601/ ,130. When T D 2, the first case of 2 predicts that α = 1 for all values of S, which is illustrated by the black region above the line T = 3,130. Furtherore, fixing the value of T 3,130, the second case of 2 predicts that α will be either one 1 if S is sall enough, or go soothly to zero as S increases. This is illustrated by any horizontal line with T 3,130. Each such line is black for sall values of S and then gradually becoes white as S increases left to right. 7 Results on sentient classification In this section we illustrate our theory on the natural language processing task of sentient classification Pang et al The point of these experients is not to instantiate the

12 Fig. 1 An illustration of the phase transition in the balance between source and target training data. The value of α iniizing the bound is indicated by the intensity, where black eans α = 1. We fix d = 1,601 and A = 0.715, approxiating the epirical setup in Fig. 3. Thex-axis shows the nuber of source instances log-scale. The y-axis shows the nuber of target instances. A phase transition occurs at 3,130 target instances. With ore target instances than this, it is ore effective to ignore even an infinite aount of source data bound fro Theore 3 directly, since the aount of data we use here is uch too sall for the bound to yield eaningful nuerical results. Instead, we show how the two ain principles of our theory fro Sects. 4 and 5 can be applied on a real-world proble. First, we show that an approxiation to the H-distance, obtained by training a linear odel to discriinate between instances fro different doains, correlates well with the loss incurred by training in one doain and testing in another. Second, we investigate iniizing the α-error as suggested by Theore 3. We show that the optial value of α for a given aount of source and target data is closely related to our approxiate H-distance. The next subsection describes the proble of sentient classification, along with our dataset, features, and learning algoriths. Then we show experientally how our approxiate H-distance is related to the adaptation perforance and the optial value of α. 7.1 Sentient classification Given a piece of text usually a review or essay, autoatic sentient classification is the task of deterining whether the sentient expressed by the text is positive or negative Pang et al. 2002; Turney While ovie reviews are the ost coonly studied doain, sentient analysis has been extended to a nuber of new doains, ranging fro stock essage boards to congressional floor debates Das and Chen 2001; Thoas et al Research results have been deployed industrially in systes that gauge arket reaction and suarize opinion fro Web pages, discussion boards, and blogs. We used the publicly available data set fro Blitzer et al. 2007b to exaine our theory. 2 The data set consists of reviews fro the Aazon website for several different types of products. We chose reviews fro the doains apparel, books, DVDs, kitchen & housewares, and electronics. Each review consists of a rating 1 5 stars, a title, and review text. We created a binary classification proble by binning reviews with 1 2 stars as negative and 4 5 stars as positive. Reviews with 3 stars were discarded. Classifying product reviews as having either positive or negative sentient fits well into our theory of doain adaptation. We note that reviews for different products have widely 2 Available at

13 Positive books review Title: A great find during an annual suer shopping trip Review: I found this novel at a bookstore on the boardwalk I visit every suer...the narrative was brilliantly told, the dialogue copletely believable and the plot totally heartwrenching. If I had ade it to the end without soe tears, I would believe yself ade of stone! Negative books review Title: The Hard Way Review: I a not sure whatever possessed e to buy this book. Honestly, it was a coplete waste of y tie. To quote a friend, it was not the best use of y entertainent dollar. If you are a fan of pedestrian writing, lack-luster plots and hackneyed character developent, this is your book. Positive kitchen & housewares review Title: no ore doggy feuds with neighbor Review: i absolutely love this product. y neighbor has four little yippers and y shepard/chow ix was antagonized by the yipping on our side of the fence. I hung the device on y side of the fence and the noise keeps the neighbors dog fro picking arguents with y dog. all barking and fighting has ceased. Negative kitchen & housewares review Title: cooks great, lid does not work well... Review: I Love the way the Tefal deep fryer cooks, however, I a returning y second one due to a defective lid closure. The lid ay close initially, but after a few uses it no longer stays closed. Since I have sall children in y hoe, I will not be purchasing this one again. Fig. 2 Soe saple product reviews for sentient classification. The top row shows reviews fro the books doain. The botto row shows reviews fro kitchen & housewares different vocabularies, so classifiers trained on one doain are likely to iss out on iportant lexical cues in a different doain. On the other hand, a single good universal sentient classifier is likely to exist naely the classifier that assigns high positive weight to all positive words and high negative weight to all negative words, regardless of product type. We illustrate the type of text in this dataset in Fig. 2, which shows one positive and one negative review each fro the doains books and kitchen & housewares. For each doain, the data set contains 1,600 labeled docuents and between 5,000 and 6,000 unlabeled docuents. We follow Pang et al and represent each instance review by a sparse vector containing the counts of its unigras and bigras, and we noralize the vectors in L 1. Finally, we discard all but the ost frequent 1,600 unigras and bigras fro each data set. In all of the learning probles of the next section, including those that require us to estiate an approxiate H-distance, we use signed linear classifiers. To estiate the paraeters of these classifiers, we iniize a Huber loss with stochastic gradient descent Zhang Experients We explore Theore 3 further by coparing its predictions to the predictions of an approxiation that can be coputed fro finite labeled source and unlabeled source and target saples. As we shall see, our approxiation is a finite-saple analog of 1. We first address λ, the error of the ideal hypothesis. Unfortunately, in general we cannot assue any relationship between the labeling functions f S and f T. Thus in order to estiate λ,weust

14 estiate ɛ T h independently of the source data. If we had enough target data to do this accurately, we would not need to adapt a source classifier in the first place. For our sentient task, however, λ is sall enough to be a negligible ter in the bound. Thus we ignore it here. We approxiate the divergence between two doains by training a linear classifier to discriinate between unlabeled instances fro the source and target doains. Then we apply Lea 2 to get an estiate of ˆd H that we denote by ζu S, U T. ζu S, U T is a lower bound on ˆd H, which is in turn a lower bound on ˆd H H.ForTheore3 to be valid, we need an upper bound on ˆd H H. Unfortunately, this is coputationally intractable for linear threshold classifiers, since finding a iniu error classifier is hard in general Ben-David et al We chose our ζu S, U T estiate because it requires no new achinery beyond an algorith for epirical risk iniization on H. Finally, we note that the unlabeled saple size is large, so we leave out the finite saple error ter for the H H-divergence. We set C to be 1,601, the VC diension of a 1,600-diensional linear classifier and ignore the log ter in the nuerator of the bound. The coplete approxiation to the bound is C α 2 fα= 1 α αζu S, U T. 3 1 C Note that in 3 corresponds to B fro 1, and ζu S, U T is a finite saple approxiation to A when λ is negligible and we have large unlabeled saples fro both the source and target doains. We copare 3 to experiental results for the sentient classification task. All of our experients use the apparel doain as the target. We obtain epirical curves for the error Fig. 3 Coparing the bound fro Theore 3 with test error for sentient classification. Each colun varies one coponent of the bound. For all plots, the y-axis shows the error and the x-axis shows α. Plots on the top row show the value given by our approxiation to the bound, and plots on the botto row show the epirical test set error. Colun a depicts different distances aong doains. Colun b depicts different nubers of target instances, and colun c represents different nubers of source instances

15 as a function of α by training a classifier using a weighted hinge loss. Suppose the target doain has weight α and there are target training instances. Then we scale the loss of target training instance by α/ and the loss of a source training instance by 1 α/1. Figure 3 shows a series of plots of 3 top row coupled with corresponding plots of test error botto row as a function of α for different aounts of source and target data and different distances between doains. In each colun, a single paraeter distance, nuber of target instances T, or nuber of source instances S is varied while the other two are held constant. Note that = T / T + S. The plots on the top row of Fig. 3 are not eant to be nuerical proxies for the true error. For the source doains books and dvd, the distance alone is well above 1/2. However, they illustrate that the bound is siilar in shape to the true error curve and that iportant relationships are preserved. Note that in every pair of plots, the epirical error curves, like the bounds, have an essentially convex shape. Furtherore, the value of α that iniizes the bound also yields low epirical error in each case. This suggests that choosing α to iniize the bound of Theore 3 and subsequently training a classifier to iniize the epirical error ˆɛ α h can work well in practice, provided we have a reasonable easure of coplexity and λ is sall. Colun a shows that ore distant source doains result in higher target error. Colun b illustrates that for ore target data, we have not only lower error in general, but also a higher iniizing α. Finally, colun c deonstrates the liitations of distant source data. When enough labeled target data exists, we always prefer to use only the target data, no atter how uch source data is available. Intuitively this is because any biased source doain cannot help to reduce error beyond soe positive constant. When the target data alone is sufficient to surpass this level of perforance, the source data ceases to be useful. Thus colun c illustrates epirically one phase transition we discuss in Sect Cobining data fro ultiple sources We now explore an extension of our theory to the case of ultiple source doains. In this setting, the learner is presented with data fro N distinct sources. Each source S j is associated with an unknown distribution D j over input points and an unknown labeling function f j. The learner receives a total of labeled saples, with j = j fro each source S j, and the objective is to use these saples to train a odel to perfor well on a target doain D T,f T, which ay or ay not be one of the sources. This setting is otivated by several doain adaptation algoriths Huang et al. 2007; Bickel et al. 2007; Jiang and Zhai 2007; Dai et al that weigh the loss fro training instances depending on how far they are fro the target doain. That is, each training instance is its own source doain. As before, we exaine algoriths that iniize convex cobinations of training error over the labeled exaples fro each source doain. Given a vector α = α 1,...,α N of doain weights with N α j = 1, we define the epirical α-weighted error of function h as N N α j ˆɛ α h = α j ˆɛ j h = hx f j x. j x S j The true α-weighted error ɛ α h is defined analogously. We use D α to denote the ixture of the N source distributions with ixing weights equal to the coponents of α. We present in turn two alternative generalizations of the bounds in Sect. 5. The first bound considers the quality and quantity of data available fro each source individually,

16 ignoring the relationships between sources. In contrast, the second bound depends directly on the H H-distance between the target doain and the weighted cobination of source doains. This dependence allows us to achieve significantly tighter bounds when there exists a ixture of sources that approxiates the target better than any single source. Both results require the derivation of unifor convergence bounds for the epirical α-error. We begin with those. 8.1 Unifor convergence The following lea provides a unifor convergence bound for the epirical α-error. Lea 6 For each j {1,...,N}, let S j be a labeled saple of size j generated by drawing j points fro D j and labeling the according to f j. For any fixed weight vector α, let ˆɛ α h be the epirical α-weighted error of soe fixed hypothesis h on this saple, and let ɛ α h be the true α-weighted error. Then for any δ 0, 1, with probability at least 1 δ: Pr [ ˆɛ α h ɛ α h ɛ ] 2ɛ 2 2exp. N αj 2 j Proof See Appendix. Note that this bound is iniized when α j = j for all j. In other words, convergence is fastest when all data instances are weighted equally. 8.2 A bound using pairwise divergence The first bound we present considers the pairwise H H-distance between each source and the target, and illustrates the trade-off that exists between iniizing the average divergence of the training data fro the target and weighting all points equally to encourage faster convergence. The ter N α j λ j that appears in this bound plays a role corresponding to λ in the previous section. Soewhat surprisingly, this ter can be sall even when there is not a single hypothesis that works well for all heavily weighted sources. Theore 4 Let H be a hypothesis space of VC diension d. For each j {1,...,N}, let S j be a labeled saple of size j generated by drawing j points fro D j and labeling the according to f j. If ĥ H is the epirical iniizer of ˆɛ α h for a fixed weight vector α on these saples and h T = in h H ɛ T h is the target error iniizer, then for any δ 0, 1, with probability at least 1 δ, ɛ T ĥ ɛ T h T + 2 N + α 2 j j d log2 logδ N α j 2λj + d H H D j,d T, where λ j = in h H {ɛ T h + ɛ j h}. 2

17 Proof See Appendix. In the special case where the H H-divergence between each source and the target is 0 and all data instances are weighted equally, the bound in Theore 4 becoes ɛ T ĥ ɛ T h T + 2 2d log log 4 δ N + 2 α j λ j. This bound is nearly identical to the ultiple source classification bound given in Theore 6 of Craer et al Aside fro the constants in the coplexity ter, the only difference is that the quantity λ i that appears here is replaced by an alternate easure of the label error between source S j and the target. Furtherore, these easures are equivalent when the true target function is a eber of H. However, the bound of Craer et al is less general. In particular, it does not handle positive H H-divergence or non-unifor weighting of the data. 8.3 A bound using cobined divergence In the previous bound, divergence between doains is easured only on pairs, so it is not necessary to have a single hypothesis that is good for every source doain. However, this bound does not give us the flexibility to take advantage of doain structure when calculating unlabeled divergence. The alternate bound given in Theore 5 allows us to effectively alter the source distribution by changing α. This has two consequences. First, we ust now deand that there exists a hypothesis h whichhaslowerroronboththeα-weighted convex cobination of sources and the target doain. Second, we easure H H-divergence between the target and a ixture of sources, rather than between the target and each single source. Theore 5 Let H be a hypothesis space of VC diension d. For each j {1,...,N}, let S j be a labeled saple of size j generated by drawing j points fro D j and labeling the according to f j. If ĥ H is the epirical iniizer of ˆɛ α h for a fixed weight vector α on these saples and h T = in h H ɛ T h is the target error iniizer, then for any δ 0, 1, with probability at least 1 δ, ɛ T ĥ ɛ T h T + 4 N α 2 j j + 2γ α + d H H D α,d T, d log2 logδ where γ α = in h {ɛ T h + ɛ α h}=in h {ɛ T h + N α j ɛ j h}. 2 Proof See Appendix. Theore 5 reduces to Theore 3 when N = 2 and one of the two source doains is the target doain that is, we have soe sall nuber of target instances.

18 8.4 Discussion One ight ask whether there exist settings where a non-unifor weighting can lead to a significantly lower value of the bound than a unifor weighting. Indeed, this can happen if soe non-unifor weighting of sources accurately approxiates the target distribution. This is true, for exaple, in the setting studied by Mansour et al. 2009a, 2009b, who derive results for cobining pre-coputed hypotheses. In particular, they show that for arbitrary convex losses, if the Rényi divergence between the target and a ixture of sources is sall, it is possible to cobine low-error source hypotheses to create a low-error target hypothesis. They then show that if for each doain j there exists a hypothesis h j with error less than ɛ, it is possible to achieve an error less than ɛ on the target by weighting the predictions of h 1,...,h N appropriately. The Rényi divergence is not directly coparable to the H H-divergence in general; however it is possible to exhibit source and target distributions which have low H Hdivergence and high even infinite Rényi divergence. For exaple, the Rényi divergence is infinite when the source and target distributions do not share support, but the H Hdivergence is only large when these regions of differing support also coincide with classifier disagreeent regions. On the other hand, we require that a single hypothesis be trained on the ixture of sources. Mansour et al. 2009a, 2009b give algoriths which do not require the original training data at all, but only a single hypothesis fro each source. 9 Conclusion We presented a theoretical investigation of the task of doain adaptation, a task in which we have a large aount of training data fro a source doain, but we wish to apply a odel in a target doain with a uch saller aount of training data. Our ain result is a unifor convergence learning bound for algoriths which iniize convex cobinations of epirical source and target error. Our bound reflects the trade-off between the size of the source data and the accuracy of the target data, and we give a siple approxiation to it that is coputable fro finite labeled and unlabeled saples. This approxiation akes correct predictions about odel test error for a sentient classification task. Our theory also extends in a straightforward anner to a ulti-source setting, which we believe helps to explain the success of recent epirical work in doain adaptation. There are two interesting open probles that deserve future exploration. First, our bounds on the divergence between source and target distribution are in ters of VC diension. We do not yet know whether our divergence easure adits tighter data-dependent bounds McAllester 2003; Bartlett and Mendelson 2002, or if there are other, ore appropriate divergence easures which do. Second, it would be interesting to investigate algoriths that choose a convex cobination of ultiple sources to iniize the bound in Theore 5 as possible approaches to adaptation fro ultiple sources. Acknowledgeents This aterial is based upon work partially supported by the Defense Advanced Research Projects Agency DARPA under Contract No. NBCHD CALO, by the National Science Foundation under grants ITR and RI , and by a gift fro Google, Inc. to the University of Pennsylvania. Koby Craer is a Horev fellow, supported by the Taub Foundations. Any opinions, findings, and conclusions or recoendations expressed in this aterial are those of the authors and do not necessarily reflect the views of the DARPA, Departent of Interior-National Business Center DOI-NBC, NSF, the Taub Foundations, or Google, Inc.

19 Open Access This article is distributed under the ters of the Creative Coons Attribution Noncoercial License which perits any noncoercial use, distribution, and reproduction in any ediu, provided the original authors and source are credited. Appendix: Proofs Theore 1 For a hypothesis h, ɛ T h ɛ S h + d 1 D S, D T + in { E DS [ fs x f T x ], E DT [ fs x f T x ]}. Proof Recall that ɛ T h = ɛ T h, f T and ɛ S h = ɛ S h, f S.Letφ S and φ T be the density functions of D S and D T respectively. ɛ T h = ɛ T h + ɛ S h ɛ S h + ɛ S h, f T ɛ S h, f T ɛ S h + ɛ S h, f T ɛ S h, f S + ɛ T h, f T ɛ S h, f T [ ɛ S h + E DS fs x f T x ] + ɛ T h, f T ɛ S h, f T [ ɛ S h + E DS fs x f T x ] + φ S x φ T x hx f T x dx ɛ S h + E DS [ fs x f T x ] + d 1 D S, D T. In the first line, we could instead choose to add and subtract ɛ T h, f S rather than ɛ S h, f T, which would result in the sae bound only with the expectation taken with respect to D T instead of D S. Choosing the saller of the two gives us the bound. Lea 2 For a syetric hypothesis class H one where for every h H, the inverse hypothesis 1 h is also in H and saples U, U of size, the epirical H-distance is [ 1 d H U, U = 2 1 in h H x:hx=0 I[x U]+ 1 x:hx=1 where I[x U] is the binary indicator variable which is 1 when x U. ] I[x U ] Proof We will show that for any hypothesis h and corresponding set Ih of positively labeled instances, [ 1 1 I[x U]+ 1 ] I[x U ] = Pr U [Ih] Pr U [Ih]. We have 1 1 = 1 2 x:hx=0 x:hx=0 x:hx=0 I [x U] + x:hx=1 x:hx=1 I [ x U ] I [x U] + I [ x U ] x:hx=1 I [x U] + I [ x U ]

20 1 = 1 2 x:hx=0 x:hx=0 I [x U] + x:hx=1 I [ x U ] I [ x U ] I [x U] x:hx=1 I [x U] I [ x U ] = Pr U [Ih] 1 Pr U [Ih] Pr U [Ih] Pr U [Ih] Mach Learn = Pr U [Ih] Pr U [Ih]. 4 The absolute value in the stateent of the lea follows fro the syetry of H. Lea 4 Let h be a hypothesis in class H. Then 1 ɛ α h ɛ T h 1 α 2 d H HD S, D T + λ. Proof Siilarly to the proof of Theore 2, this proof relies heavily on the triangle inequality for classification error. ɛ α h ɛ T h = 1 α ɛ S h ɛ T h 1 α [ ɛ S h ɛ S h, h + ɛ S h, h ɛ T h, h + ɛ T h, h ɛ T h ] 1 α [ ɛ S h + ɛ S h, h ɛ T h, h +ɛ T h ] 1 1 α 2 d H HD S, D T + λ. Theore 3 Let H be a hypothesis space of VC diension d. Let U S and U T be unlabeled saples of size each, drawn fro D S and D T respectively. Let S be a labeled saple of size generated by drawing points fro D T and 1 points fro D S, labeling the according to f S and f T, respectively. If ĥ H is the epirical iniizer of ˆɛ α h on S and h T = in h H ɛ T h is the target error iniizer, then for any δ 0, 1, with probability at least 1 δ over the choice of the saples, ɛ T ĥ ɛ T h T + 4 α α α 2 ˆd H H U S, U T + 4 2d log log 8 δ 2d log2 + log 4 δ + λ. Proof The coplete proof is ostly identical to the standard proof of unifor convergence for epirical risk iniizers. We show here the steps that are different. Below we use L4, and Th2 to indicate that a line of the proof follows by application of Lea 4 or Theore 2 respectively. L5 indicates that the proof follows by Lea 5, but also relies on saple syetrization and bounding the growth function by the VC diension Anthony