Rotational Prior Knowledge for SVMs

Transcription

1 Rotational Prior Knowledge for SVMs Arkady Epshteyn and Gerald DeJong University of Illinois at Urbana-Chapaign, Urbana, IL 68, USA Abstract. Incorporation of prior knowledge into the learning process can significantly iprove low-saple classification accuracy. We show how to introduce prior knowledge into linear support vector achines in for of constraints on the rotation of the noral to the separating hyperplane. Such knowledge frequently arises naturally, e.g., as inhibitory and excitatory influences of input variables. We deonstrate that the generalization ability of rotationally-constrained classifiers is iproved by analyzing their VC and fat-shattering diensions. Interestingly, the analysis shows that large-argin classification fraework justifies the use of stronger prior knowledge than the traditional VC fraework. Epirical experients with text categorization and political party affiliation prediction confir the usefulness of rotational prior knowledge. Introduction Support vector achines (SVMs) have outperfored copeting classifiers on any classification tasks [,2,]. However, the aount of labeled data needed for SVM training can be prohibitively large for soe doains. Intelligent user interfaces, for exaple, ust adopt to the behavior of an individual user after a liited aount of interaction in order to be useful. Medical systes diagnosing rare diseases have to generalize well after seeing very few exaples. Natural language processing systes learning to identify infrequent social events (e.g., revolutions, wars, etc.) fro news articles have access to very few training exaples. Moreover, they rely on anually labeled data for training, and such data is often expensive to obtain. Various techniques have been proposed specifically to deal with the proble of learning fro very sall datasets. These include active learning [4], hybrid generative-discriinative classification [], learning-to-learn by extracting coon inforation fro related learning tasks [6], and using prior knowledge. In this work, we focus on the proble of using prior knowledge to increase the accuracy of a large argin classifier at low saple sizes. Several studies have shown the efficacy of this ethod. Scholkopf et. al. [7] deonstrate how to integrate prior knowledge about invariance under transforations and iportance of local structure into the kernel function. Fung et. al. [8] use doain knowledge in for of labeled polyhedral sets to augent the training data. Wu and Srihari [9] allow huan users to specify their confidence in the exaple s label, varying the effect of each exaple on the separating hyperplane proportionately

2 to its confidence. Mangasarian et. al. [] introduce prior knowledge into the large-argin regression fraework. While the ability of prior knowledge to iprove any classifier s generalization perforance is well-known, the properties of large argin classifiers with prior knowledge are not well understood. In order to study this proble, we introduce a new for of prior knowledge for SVMs (rotational constraints) and prove that it is possible to obtain stronger guarantees for the generalization ability of constrained classifiers in the large-argin fraework than in the classical VC fraework. Specifically, we show that the VC diension of our classifier reains large even when its hypothesis space is severely constrained by prior knowledge. The fat-shattering diension, however, continues to decrease with decreasing hypothesis space, justifying the use of stronger doain knowledge. We conduct experients to deonstrate iproveents in perforance due to rotational prior knowledge and copare the with iproveents achievable by active learning. 2 Preliinaries The SVM classifier with a linear kernel learns a function of the for sign(f(x; ω, θ) = ω T x + θ = n ω i x i + θ) () i = that aps (x; ω, θ) R n xw, Θ to one of the two possible output labels {, }. Given a training saple of points (x, y )...(x, y ), SVM seeks to axiize the argin between the separating hyperplane and the points closest to it []. For canonical hyperplanes (i.e., hyperplanes with unit argins), the axiu-argin hyperplane iniizes the regularized risk functional R reg [f, l] = l(y i, f(x i ; ω, θ)) + C 2 ω 2 2 (2) i = with hard argin - loss given by l(y i, f(x i ; ω, θ)) = I { yi f(x i ;ω,θ)>}. The soft argin forulation allows for deviation fro the objective of axiizing the argin in order to better fit the data. This is done by substituting the hinge loss function l(y i, f(x i ; ω, θ)) = ax( y i f(x i ; ω, θ), ) into (2). Miniizing the regularized risk (2) in the soft argin case is equivalent to solving the following (prial) optiization proble: iniize ω, θ, ξ 2 ω C ξ i subj. to y i = i (ω T x + θ) ξ i, i =... () sign(y) = if y, otherwise

3 Calculating the Wolfe dual fro () and solving the resulting axiization proble: axiize α i 2 α i α j y i y j (x T i x j) (4) α i = i, j = subject to C α i, i =... and α i y i = i = yields the solution ω = α i y i x i () i = Setting ξ i =, i =..., (), (4), and () can be used to define and solve the original hard argin optiization proble. The generalization error of a classifier is governed by its VC diension []: Definition. A set of points S = {x...x } is shattered by a set of functions F apping fro a doain X to {, } if, for each b {, }, there is a function f b in F with bf b (x i ) =, i =... The VC-diension of F is the cardinality of the largest shattered set S. Alternatively, the fat-shattering diension can be used to bound the generalization error of a large argin classifier[]: Definition 2. A set of points S = {x...x } is γ-shattered by a set of functions F apping fro a doain X to R if there are real nubers r,..., r such that, for each b {, }, there is a function f b in F with b(f b (x i ) r i ) γ, i =... We say that r,..., r witness the shattering. Then the fat-shattering diension of F is a function fat F (γ) that aps γ to the cardinality of the largest γ-shattered set S. Proble Forulation and Generalization Error In this work, we introduce prior knowledge which has not been previously applied in the SVM fraework. This prior is specified in ters of explicit constraints placed on the noral vector of the separating hyperplane. For exaple, consider the task of deterining whether a posting cae fro the newsgroup alt.atheis or talk.politics.guns, based on the presence of the words gun and atheis in the posting. Consider the unthresholded perceptron f(posting; ω atheis, ω gun, θ) = ω atheis I {atheis present} + ω gun I {gun present} + θ (I {x present} is the indicator function that is when the word x is present in the posting and otherwise). A positive value of ω atheis captures excitatory influence of the word atheis on the outcoe of classification by ensuring that the value of f(posting; ω atheis, ω gun, θ) increases when the word atheis is encountered in the posting, all other things being equal. Siilarly, constraining ω gun to be

4 negative captures an inhibitory influence. Note that such constraints restrict the rotation of the hyperplane, but not its translation offset θ. Thus, prior knowledge by itself does not deterine the decision boundary. However, it does restrict the hypothesis space. We are interested in iposing constraints on the paraeters of the faily F of functions sign(f(x; ω, θ)) defined by (). Constraints of the for ω T c > generalize excitatory and inhibitory sign constraints 2 (e.g., ω i > is given by c = [c =,..., c i =,..., c n = ] T ). In addition, soeties it is possible to deterine the approxiate orientation of the hyperplane a-priori. Noralizing all the coefficients ω i in the range [, ] enables the doain expert to specify the strength of the contribution of ω gun and ω atheis in addition to to the signs of their influence. When prior knowledge is specified in ters of an orientation vector v, the conic constraint fro deviating too far fro v. ω T v ω v > ρ (ρ [, )) prevents the noral ω It is well-known that the VC-diension of F in R n is n + (see, e.g., [2]). Interestingly, the VC-diension of constrained F is at least n with any nuber of constraints iposed on ω W as long as there is an open subset of W that satisfies the constraints (this result follows fro []). This eans that any value of ρ in the conic constraint cannot result in significant iproveent in the classifier s generalization ability as easured by its VC-diension. Siilarly, sign constraints placed on all the input variables cannot decrease the classifier s VCdiension by ore than. The following theore shows that the VC-diension of a relatively weakly constrained classifier achieves this lower bound of n: Theore. For the class F C n = {x sign( ω i x i + θ) : ω > }, VC- i = diension of F C = n. Proof. The proof uses techniques fro [2]. Let F C = {x sign(ω x + ω T x + θ) : ω > }, where x = [x 2,..., x n ] T is the projection of x into the hyperplane {ω = } and ω = [ω 2,...ω n ] T. First, observe that {ω > } defines an open subset of W. Hence, the VC-diension of F C is at least n. Now, we show by contradiction that a set of n + points cannot be shattered by F C. Assue that soe set of points x,..., x n+ R n can be shattered. Let x,..., x n+ R n be their projections into the hyperplane {ω = }. There are two cases: Case : x,..., x n+ are distinct. Since these are n + points in an (n )- diensional hyperplane, by Radon s Theore [4] they can be divided into two sets S and S 2 whose convex hulls intersect. Thus, λ i, λ j ( λ i, λ j ) 2 In the rest of the paper, we refer to excitatory and inhibitory constraints of the for ω i > (ω i < ) as sign constraints because they constrain the sign of w i.

5 such that λ i x i = i : x i S j : x j S 2 λ j x j (6) and i : x i S λ i = j : x j S 2 λ j = (7) Since x,..., x n+ are shattered in R n, ω, ω, θ such that ω x i + ω T x i θ for all x i S. Multiplying by λ i and suing over i, we get (after applying (7)) ω T λ i x i θ ω λ i x i (8) i : x i S i : x i S Siilarly, for all x j S 2, ω x j + ωt x j < θ ω T λ j x j < θ ω λ j x j j : x j S 2 j : x j (9) S 2 Cobining (8), (9), and (6) yields ω ( λ j x j j : x j λ i x i ) < S 2 i : x i S () Since ω >, ( λ j x j j : x j λ i x i ) < () S 2 i : x i S Now, shattering the sae set of points, but reversing the labels of S and S 2 iplies that ω, ω, θ such that ω x i + ω T x i < θ for all x i S and ω x j + ω T x j θ for all x j S 2. An arguent identical to the one above shows that ω ( λ j x j j : x j λ i x i ) > (2) S 2 i : x i S Since ω >, ( λ j x j j : x j λ i x i ) >, which contradicts () S 2 i : x i S Case 2: Two distinct points x and x 2 project to the sae point x = x 2 () on the hyperplane {ω = }. Assue, wlog, that x < x 2 (4). Since x and x 2 are shattered, ω, ω, θ such that ω x + ω T x θ > ω x 2 + ω T x 2, which, together with () and (4), iplies that ω <, a contradiction.

6 This result eans that iposing a sign constraint on a single input variable or using ρ = in the conic constraint is sufficient to achieve the axiu theoretical iproveent within the VC fraework. However, it is unsatisfactory in a sense that it contradicts our intuition (and epirical results) which suggests that stronger prior knowledge should help the classifier reduce its generalization error faster. The following theore shows that the fat-shattering diension decreases continuously with increasing ρ in the conic constraint, giving us the desired guarantee. Technically, the fat-shattering diension is a function of the argin γ, so we use the following definition of function doination to specify what we ean by decreasing fat-shattering diension: Definition. A function f (x) is doinated by a function f 2 (x) if, for all x, f (x) f 2 (x) and, at least for one a, f (a) < f 2 (a). When we say that f ρ (x) decreases with increasing ρ, we ean that ρ < ρ 2 iplies that f ρ2 (x) is doinated by f ρ (x). Theore 2. For the class F v,ρ = {x ω T x + θ : ω 2 =, v 2 =, x 2 R, ω T v > ρ }, fat Fv,ρ (γ) decreases with increasing ρ. 4 Proof. The fat-shattering diension obviously cannot increase with increasing ρ, so we only need to find a value of γ where it decreases. We show that this happens at γ = R ρ 2 2. First, we upper bound fat F v,ρ2 (γ ) by showing that, in order to γ -shatter two points, the separating hyperplane ust be able to rotate through a larger angle than that allowed by the constraint ω T v > ρ 2. Assue that two points x, x 2 can be γ -shattered by F v,ρ2. Then ω, ω 2, θ, θ 2, r, r 2 such that ω T x + θ r γ, ω T x 2 + θ r 2 γ, ω 2T x + θ 2 r γ, ω 2T x 2 + θ 2 r 2 γ. Cobining the ters and applying the Cauchy-Schwartz inequality, we get ω ω 2 2γ R. Squaring both sides, expanding ω ω 2 2 as ω 2 + ω 2 2 2ω T ω 2, and using the fact that ω = ω 2 = yields ω T ω 2 2γ 2 R 2 = 2ρ2 2 () Since the angle between ω and ω 2 cannot exceed the su of the angle between ω and the prior v and the angle between v and ω 2, both of which are bounded above by arccos(ρ 2 ), we get (after soe algebra) ω T ω 2 > 2ρ 2 2, which contradicts (). Thus, fat Fv,ρ2 (R ρ 2 2 ) < 2 (6) Now, we lower bound fat Fv,ρ (γ ) by exhibiting two points γ -shattered by F v,ρ. Wlog, let v = [,,,..] T. It is easy to verify that x = [R,,..] T and x 2 = The constraint {w > } is weak since it only cuts the volue of the hypothesis space by. 4 2 Note that the stateent of this theore deals with hyperplanes with unit norals, not canonical hyperplanes. The argin of a unit-noral hyperplane is given by in i=.. ωx i + θ.

7 [ R,,..] T can be R ρ 2 2 -shattered by F v,ρ, witnessed by r = r 2 =. Hence, fat Fv,ρ (R ρ 2 2 ) 2 (7) which, cobined with (6), copletes the arguent. The result of Theore is iportant because it shows that even weak prior knowledge iproves the classifier s generalization perforance in the VC fraework which akes less assuptions about the data than the fat-shattering fraework. However, it is the result of Theore 2 within the fat-shattering fraework which justifies the use of stronger prior knowledge. 4 Ipleentation The quadratic optiization proble for finding the axiu argin separating hyperplane (2) can be easily odified to take into account linear rotational constraints of the for ω T c j >, j =...l. The soft argin/soft constraint forulation that allows for possibility of violating both the argin axiization objective and the rotational constraints iniizes the following regularization functional: R reg [f, l, l ] = l(y i, f(x i ; ω, θ)) + C C i = 2 l l j = l (ω, c j ) + C 2 ω 2 2 (8) with - losses for the data and the prior: l(y i, f(x i ; ω, θ)) = I { yi f(x i ;ω,θ)>} and l (ω, c j ) = I { ω T c j >} in the hard argin/hard rotational constraints case and hinge losses: l(y i, f(x i ; ω, θ)) = ax( y i f(x i ; ω, θ), ),l (ω, c j ) = ax( ω T c j, ) in the soft argin/soft rotational constraints case. The regularization functional above is the sae as in (2) with an additional loss function which penalizes the hyperplanes that violate the prior. Miniizing (8) with hinge loss functions is equivalent to solving: iniize ω, θ, ξ, ν 2 ω C i = ξ i + C 2 l l ν j (9) j = subject to y i (ω T x + θ) ξ i, ξ i, i =..., ω T c j ν j, ν j, j =...l. Constructing the Lagrangian fro (9) and calculating the Wolfe dual results in the following axiization proble: axiize α, β α i 2 α i α j y i y j (x T i x j) i = i, j = l α i β j y i (x T i c j ) l β i β j (x T i c j ) (2) 2 i = j = i, j =

8 a) b) c) Fig.. Approxiating a conic constraint: a) Start with the known constraints ω, ω 2, and ω around v = [,, ] T. The figure shows linear constraints around v (white vector) and the cone ωt v ω > approxiated by these constraints. b) Rotate the bounding hyperplanes {ω = }, {ω 2 = }, {ω = } into v, approxiating a cone with the required angle ρ around v c) Rotate the whole boundary fro v (white vector in (a),(b)) to the required orientation v (white vector in (c)). subj. to C α i, i =..., C 2 l β j, i =...l, and α i y i = i = The solution to (2) is given by ω = l α i y i x i + β j c j. (2) i = j = As before, setting ξ i =, ν j =, i =..., j =...l, (9), (2), and (2) can be used to solve the hard argin/hard rotational constraints optiization proble. Note that in the soft-argin forulation, constants C and C 2 define a trade-off between fitting the data, axiizing the argin, and respecting the rotational constraints. The above calculation can ipose linear constraints on the orientation of the large argin separating hyperplane when such constraints are given. This is the case with sign-constrained prior knowledge. However, doain knowledge in for of a cone centered around an arbitrary rotational vector v cannot be represented as a linear constraint in the quadratic optiization proble given by (9). The approach taken in this work is to approxiate an n-diensional cone with n hyperplanes. For exaple, sign constraints ω, ω 2, and ω approxiate a cone of angle ρ = around v = [,, ] T (see Figure -(a)). To approxiate a cone of arbitrary angle ρ around an arbitrary orientation vector v, ) the noral ω i of each bounding hyperplane {ω i = } (as defined by the sign constraints above) is rotated in the plane spanned by {ω i, v } by an angle acos(ω i T v ) ρ, and 2) a solid body rotation that transfors v into v is subsequently applied to all the bounding hyperplanes, as illustrated in Figure. This construction generalizes in a straightforward way fro R to R n.

9 Experients a) b) Deocrat (ω i > ).8 Handicapped Infants.6 Anti-Satellite Weapon Test Ban.4 Aid To Nicaraguan Contras.2 Iigration South Africa Export Adinistration Act. Republican (ω i < ).8 Military Aid to El Salvador.6 Religious Groups in Schools.4 Generalization Error Voting sv w/prior sv Nuber of Training Points Fig. 2. a) Prior Knowledge for voting b)generalization error as a percentage versus nuber of training points for voting classification. For each classification task, the data set is split randoly into training and test sets in different ways. SVM classifier is trained on the training set with and without prior knowledge, and its average error on the test set is plotted, along with error bars showing 9% confidence intervals. Experients were perfored on two distinct real-world doains: Voting Records. This is a UCI database [] of congressional voting records. The vote of each representative is recorded on the 6 key issues. The task is to predict the representative s political party (Deocrat or Republican) based on his/her votes. The doain theory was specified in for of inhibitory/excitatory sign constraints. An excitatory constraint eans that the vote of yea correlates with the Deocratic position on the issue, an inhibitory constraint eans that Republicans favor the proposal. The coplete doain theory is specified in Figure 2-(a). Note that sign constraints are iposed on relatively few features (7 out of 6). Since this type of doain knowledge is weak, a hard rotational constraint SVM was used. Only representatives whose positions are known on all the 6 issues were used in this experient. The results shown in Figure 2- (b) deonstrate that sign constraints decrease the generalization error of the classifier. As expected, prior knowledge helps ore when the data is scarce. Text classification. The task is to deterine the newsgroup that a posting was taken fro based on the posting s content. We used the 2-newsgroups dataset [6]. Each posting was treated as a bag-of-words, with each binary feature encoding whether or not the word is present in the posting. Steing was used in the preprocessing stage to reduce the nuber of features. Feature selection based on utual inforation between each individual feature and the label was eployed ( axially inforative features were chosen). Since SVMs are best suited for binary classification tasks, all of our experients involve pairwise newsgroup classification. The proble of applying SVMs to ulticat-

10 egory classification has been researched extensively([2,]), and is orthogonal to our work. a)atheis vs politics.guns b)politics. guns vs ideast c)religion.christian vs autos d)sci.edicine vs sport.hockey e) sci.edicine vs politics.guns Legend: sv w/prior sv active sv Fig.. Generalization error as a percentage versus nuber of training points for different classification experients. For each rando saple selection, the data set is split randoly into training and test sets in different ways. For active learning experients, the data set is split randoly into two equal-sized sets in different ways, with one set used as the unlabeled pool for query selection, and the other set - for testing. All error bars are based on 9% confidence intervals. Prior knowledge in this experient is represented by a conic constraint around a specific orientation vector v. While it ay be hard for huan experts to supply such a prior, there are readily available sources of doain knowledge that were not developed specifically for the classification task at hand. In order to be able to utilize the, it is essential to decode the inforation into a for usable by the learning algorith. This is the virtue of rotational constraints: they are directly usable by SVMs and they can approxiate ore sophisticated pre-existing fors of inforation. In our experients, doain knwoledge fro Wordnet, a lexical syste which encodes seantic relations between words [7], is autoatically converted into v. The coefficient v x of each word x is calculated fro the relative proxiity of x to each category label in the hyperny (is-a) hierarchy of Wordnet (easured in hops). A natural approxiation of v x is given hops(x,label +) hops(x,label )+hops(x,label + ) by, noralized by a linear apping to the required range [, ], where label + and label are the naes of the two newsgroups. Perforance of the following three classifiers on this task was evaluated:

11 . A soft rotational constraint SVM (C = C 2 = ) with Wordnet prior (ρ =.99) (reasonable values of constants were picked based on the alt.atheis vs. politics.guns classification task, with no attept to optiize the for other tasks). 2. An SVM which actively selects the points to be labeled out of a pool of unlabeled newsgroup postings. We ipleented a strategy suggested in [4] which always queries the point closest to the separating hyperplane.. Traditional SVM (C = ) trained on a randoly selected saple. Typical results of this experient for a few different pairwise classification tasks appear in Figure. For sall data saples, the prior consistently decreases generalization error by up to 2%, showing that even a very approxiate prior orientation vector v can result in significant perforance iproveent. Since prior knowledge is iposed with soft constraints, the data overwhels the prior with increasing saple size. Figure also copares the effect of introducing rotational constraints with the effect of active learning. It has been shown theoretically that active learning can iprove the convergence rate of the classification error under a favorable distribution of the input data [8], although no such guarantees exist for general distributions. In our experients, active learning begins to iprove perforance only after enough data is collected. Active learning does not help when the saple size is very sall, probably due to the fact that the separating hyperplane of the classifier cannot be approxiated well, resulting in uninforative choices of query points. Rotational prior knowledge, on the other hand, is ore helpful for lowest saple sizes and ceases to be useful in the region where active learning helps. Thus, the strengths of prior knowledge and active learning are copleentary. Cobining the is a direction for future research. 6 Conclusions. We presented a siple fraework for incorporating rotational prior knowledge into support vector achines. This fraework has proven not only practically useful, but also useful for gaining insight into generalization ability of a-priori constrained large-argin classifiers. Related work includes using Wordnet for feature creation for text categorization ([9]) and introducing sign constraints into the perceptron learning algorith [2,2]. These studies do not provide generalization error guarantees for classification. Acknowledgeents. We thank Ilya Shpitser and anonyous reviewers for helpful suggestions on iproving this paper. This aterial is based upon work supported in part by the National Science Foundation under Award NSF CCR -24 ITR and in part by the Inforation Processing Technology Office of the Defense Advanced Research Projects Agency under award HR---4. Any opinions, findings, and conclusions or recoendations expressed in this publication are those of the authors and do not necessarily reflect the views of the National Science Foundation or the Defense Advanced Research Projects Agency.

12 References. Vapnik, V.: The Nature of Statistical Learning Theory. Springer-Verlag (99) 2. Joachis, T.: Text categorization with support vector achines: learning with any relevant features. In: Proceedings of the Tenth European Conference on Machine Learning. Nuber 98 (998). Duas, S., Platt, J., Heckeran, D., Sahai, M.: Inductive learning algoriths and representations for text categorization. Proceedings of the Seventh International Conference on Inforation and Knowledge Manageent (998) 4. Capbell, C., Cristianini, N., Sola, A.: Query learning with large argin classifiers. Proceedings of The Seventeenth International Conference on Machine Learning (2) 8. Raina, R., Shen, Y., Ng, A., McCallu, A.: Classification with hybrid generative/discriinative odels. Proceedings of the Seventeenth Annual Conference on Neural Inforation Processing Systes (2) 6. Fink, M.: Object classification fro a single exaple utilizing class relevance etrics. Proceedings of the Eighteenth Annual Conference on Neural Inforation Processing Systes (24) 7. Scholkopf, B., Siard, P., Vapnik, V., Sola, A.: Prior knowledge in support vector kernels. Advances in kernel ethods - support vector learning (22) 8. Fung, G., Mangasarian, O., Shavlik, J.: Knowledge-based support vector achine classifiers. Proceedings of the Sixteenth Annual Conference on Neural Inforation Processing Systes (22) 9. Wu, X., Srihari, R.: Incorporating prior knowledge with weighted argin support vector achines. Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (24). Mangasarian, O., Shavlik, J., Wild, E.: Knowledge-based kernel approxiation. Journal of Machine Learning Research (24). Shawe-Taylor, J., Bartlett, P.L., Williason, R.C., Anthony, M.: Structural risk iniization over data-dependent hierarchies. IEEE Transactions on Inforation Theory 44 (998) 2. Anthony, M., Biggs, N.: PAC learning and artificial neural networks. Technical report (2). Erlich, Y., Chazan, D., Petrack, S., Levy, A.: Lower bound on VC-diension by local shattering. Neural Coputation 9 (997) 4. Grunbau, B.: Convex Polytopes. John Wiley (967). Blake, C., Merz, C.: UCI repository of achine learning databases, learn/lrepository.htl (998) 6. Blake, C., Merz, C.: 2 newsgroups database, 2newsgroups/ (998) 7. Miller, G.: WordNet: an online lexical database. International Journal of Lexicography (99) 8. Dasgupta, S., Kalai, A.T., Monteleoni, C.: Analysis of perceptron-based active learning. Eighteenth Annual Conference on Learning Theory (2) 9. Gabrilovich, E., Markovitch, S.: Text categorization with any redundant features: Using aggressive feature selection to ake svs copetitive with c4.. Proceedings of The Twenty-First International Conference on Machine Learning (24) 2. Ait, D., Capbell, C., Wong, K.: The interaction space of neural networks with sign-constrained weights. Journal of Physics (989) 2. Barber, D., Saad, D.: Does extra knowledge necessarily iprove generalization? Neural Coputation 8 (996)