An Ensemble Approach for Incremental Learning in Nonstationary Environments

Transcription

1 An Ensemble Approach for Incremenal Learning in Nonsaionary Environmens Michael D. Muhlbaier and Robi Polikar * Signal Processing and Paern Recogniion laboraory Elecrical and Compuer Engineering, Rowan Universiy, Glassboro, NJ 828 USA muhlbaier@ieee.org, polikar@rowan.edu Absrac. We describe an ensemble of classifiers based algorihm for incremenal learning in nonsaionary environmens. In his formulaion, we assume ha he learner is presened wih a series of raining daases, each of which is drawn from a differen snapsho of a disribuion ha is drifing a an unknown rae. Furhermore, we assume ha he algorihm mus learn he new environmen in an incremenal manner, ha is, wihou having access o previously available daa. Insead of a ime window over incoming insances, or an aged based forgeing as used by mos ensemble based nonsaionary learning algorihms a sraegic weighing mechanism is employed ha racks he classifiers performances over drifing environmens o deermine appropriae voing weighs. Specifically, he proposed approach generaes a single classifier for each daase ha becomes available, and hen combines hem hrough a dynamically modified weighed majoriy voing, where he voing weighs hemselves are compued as weighed averages of classifiers individual performances over all environmens. We describe he implemenaion deails of his approach, as well as is iniial resuls on simulaed non-saionary environmens. Keywords: Nonsaionary environmen, concep drif, Learn ++.NSE. Inroducion The problem of learning in nonsaionary environmens (NSE) has radiionally received much less aenion han is saionary counerpar. I is perhaps due o inheren difficuly of he problem: afer all, machine learning algorihms require a formal and precise definiion of he problem, and in his case, i is even difficul o define he problem: wha exacly is a nonsaionary environmen? Informally, i refers o he variaion of he underlying daa disribuion ha defines he concep o be learned: he environmen subsequenly provides new daa, for which he inpu/oupu mapping (decision boundary) is differen han hose of he previous daases. Early work in NSE learning, also known as concep drif, have concenraed on formalizing exacly wha kind of drif and/or how fas of a drif can be learned [-4]. Therein lies he difficuly wih his problem; change can be slow or fas, abrup or gradual, random or sysemaic, cyclical, expanding or conracing in he feaure space. * Corresponding auhor. M. Haindl, J. Kiler, and F. Roli (Eds.): MCS 27, LNCS 4472, pp. 49, 27. Springer-Verlag Berlin Heidelberg 27

2 An Ensemble Approach for Incremenal Learning in Nonsaionary Environmens 49 Several approaches have been proposed for various combinaions of such changes, which ypically include a mechanism o (i) deec he drif and/or is magniude; (ii) learn he change in he environmen; and/or (iii) forge wha is no longer relevan. Many algorihms specifically designed for concep drif are generally based in par on he ideas used in FLORA [3]: use a iming window (fixed or variable) o choose a block of (new) insances as hey come in, and hen rain a new classifier. The window size can be increased or decreased depending on how fas he environmen is changing. Of course, such algorihms have a buil-in forgeing mechanism wih he implici assumpion ha hose insances ha fall ouside of he window are no longer relevan, and he informaion carried by hem can be forgoen. Oher approaches ry o deec when a subsanial change has occurred (novely deecion), and hen updae he classifier [-8], or find he exreme examples ha carry mos relevan and novel informaion o rain a classifier (parial insance memory) [9]; sill ohers deec and purge hose insances ha no longer carry relevan informaion, and rain a new classifier wih he remaining block of daa []. A differen group of approaches rea he nonsaionary learning as a predicion problem, and use appropriae classificaion (e.g., PNN) [] or racking algorihms (such as Kalman filers) [2]. More recenly, ensemble based approaches have also been proposed. As reviewed by Kuncheva in [3], such algorihms ypically fall ino one of he following caegories: (i) dynamic combiners where, for a previously rained fixed ensemble, he combinaion rule is changed o rack he concep drif, e.g., weighed majoriy voing or Winnow based algorihms [4]; (ii) algorihms ha use he new raining o updae an online learner or all members of an ensemble, e.g. online boosing []; and (iii) algorihms ha add new ensemble members [6] and/or replace he leas conribuing member of an ensemble wih a classifier rained on new daa [7,8]. In his paper we propose an alernaive formulaion, Learn ++.NSE, buil upon our previously inroduced incremenal learning algorihm Learn ++ [9], by suiably modifying i for nonsaionary environmens. Learn ++.NSE does no compleely fi ino any of he above caegories, bu raher combines ideas from each. I does use new daa o creae new ensemble members, and i also adjuss he combinaion rule by dynamically modifying voing weighs. I does no use a iming window, or any of he previously seen daa (hence an incremenal learning approach), and i does no discard old classifiers, in case old classifiers become relevan again in he fuure. I simply reweighs hem based on heir prediced experise on he curren environmen. I is reasonable o quesion wheher here is need for ye anoher ensemble based algorihm for nonsaionary learning, in paricular considering ha we make no claim on he superioriy of Learn ++.NSE over any of he oher algorihms ensemble based or oherwise. In he spiri of he no-free-lunch heorem, we believe ha no single algorihm can ouperform all ohers on all applicaions, ha he success depends much on he mach of he characerisics of he algorihm wih hose of he problem, and herefore i is bes o have access o a oolbox of algorihms each wih is own paricular srenghs and weaknesses. Considering he differences of Learn ++.NSE menioned above, along wih he promising iniial resuls and oucomes discussed laer in his paper, we believe ha Learn ++.NSE can be a beneficial alernaive on a variey of nonsaionary learning scenarios. The algorihm is described in deail in Secion 2,

3 492 M.D. Muhlbaier and R. Polikar followed by a descripion of our simulaion ess and resuls in Secion 3. Specific nonsaionary environmen scenarios in which he algorihm is expeced o be paricularly useful are discussed in Secion 4. 2 Learn ++.NSE Learn ++.NSE uses a similar srucural framework as Learn ++ : incremenally build an ensemble of classifiers from new daa ha are combined by weighed majoriy voing, and he exising ensemble decides on he conribuion of each successive classifier. There are a number of differences, however, insiued for suiably modifying he algorihm for nonsaionary environmens. In summary, we assume ha each new daase represens a new snapsho of he environmen. The amoun of change in he environmen since he previous daase may be minor or subsanial, and is racked by he performance of he exising ensemble on he curren daase. Learn ++.NSE creaes a single classifier for each daase ha becomes available from he new environmen, and a weighed performance measure (based on is error) is compued for each classifier on each environmen i has experienced. Each performance measure represens he experise of ha classifier on a paricular snapsho of he environmen. These performance measures are hen averaged using a nonlinear (sigmoidal) funcion, giving higher weighs o he performances of he classifier on he more recen environmens. A any ime, he classifiers can be combined hrough weighed majoriy voing, using he mos recen averaged weighs, o obain he final hypohesis. In Learn ++.NSE, he change in he environmen is racked boh by addiion of new classifiers, as well as he voing weighs of exising classifiers, and no classifier is ever discarded. If cyclical drif makes earlier classifiers once again relevan, he algorihm recognizes his change, and assigns higher weighs o earlier classifiers. The algorihm is described in deail below, wih is pseudocode given in Fig.. We assume ha an oracle provides us wih a raining daase a cerain inervals, (no necessarily uniform). A each inerval, he disribuion from which he oracle draws he raining daa changes in some manner and rae, also unknown o us. The raining daase D of cardinaliy m a ime provides us wih a snapsho of he hen curren environmen. Learn ++.NSE generaes one classifier for each such new daase ha becomes available. To do so, he algorihm firs evaluaes he classificaion accuracy of he currenly available composie hypohesis H - on he newly available daa D. H -, obained by he weighed majoriy voing of all classifiers generaed during he previous - ieraions, represens he exising ensemble s knowledge of he environmen. The error of he composie hypohesis E is compued as a simple raio of he correcly idenified insances of he new daase. We demand ha his error be less ha a hreshold, say ½, o ensure ha i has a meaningful classificaion capaciy. For his selecion of he hreshold, we normalize he error so ha he normalized error B remains beween and (Sep inside he Do loop). We hen updae a se of weighs for each insance, ha is normally iniialized o be uniform (/m ): he weighs of he insances misclassified by he ensemble are reduced by a facor of B. The weighs are hen normalized o obain a disribuion D (sep 2).

4 An Ensemble Approach for Incremenal Learning in Nonsaionary Environmens 493 Inpu: For each daase D =,2, represening a new environmen Sequence of i=,,m insances x i wih labels y i Y = {,,c} Supervised learning algorihm BaseClassifier. Do for =,2, D i = w i = / m, i, skip o sep 3. () If = Iniialize () (). Compue error of exising ensemble on new daa m ( ) ( ) i= i Normalize error B E ( E ) i E = m H x y (2) 2. Updae insance weighs = (3) ( ) = wi = m, oherwise Se B, H xi yi m i i= D = w w so ha Dk is a disribuion. () 3. Call BaseClassifier wih D, obain h : X Y 4. Evaluae all exising classifiers on new daase D m () ( ) = D i h x y, for k =,..., (6) ε = >, generae a new h. If ε < > 2, se ε = 2 ε k i= k i i If k 2 ( ) k βk = εk εk, for k =,..., (7). Compue a weighed sum of all normalized errors for k h classifier h k a ( k b) k j ( ) ω k = + e, k j j k k k j= β = ω β, for k =,..., k k j= k k (4) ω = ω ω (8) (9) 6. Calculae classifier voing weighs ( βk) W = log, for k =,..., () k 7. Obain he composie hypohesis as he curren final hypohesis ( ) arg max ( ) H x i = W k k h x c k i = c () Fig.. The pseudocode of he algorihm Learn ++.NSE The algorihm hen calls he BaseClassifier and asks i o creae he h classifier h using daa drawn from he curren raining daase D provided by he oracle (sep 3). All classifiers generaed hus far h k, k=,..., are hen evaluaed on he curren daase, by compuing heir weighed error (Equaion 6, sep 4). Noe ha a curren ime sep, we

5 494 M.D. Muhlbaier and R. Polikar now have error measures, one for each classifier generaed hus far. Hence he error erm ε k represens he error of he k h classifier h k a he h ime sep. The errors are again assumed o be less han ½. Here, we make a disincion: if he error of he mos recen classifier (on is naive raining se) is greaer han ½, hen we discard ha classifier and generae a new one. For any of he oher (older) classifiers, if is error is greaer han ½, i is se o ½. This effecively ses he normalized error of ha classifier a ha ime sep o, which in urn removes all of is voing power laer during he weighed majoriy voing. Noe ha unlike AdaBoos and similar algorihms ha use such an error hreshold, Learn ++.NSE does no abor, nor does i hrow away (he previously generaed) classifiers when heir error exceeds ½. This is because, i is no unreasonable for a classifier o perform poorly if he environmen has changed drasically since i was creaed, and furhermore, i does no mean ha his classifier will never be useful again in he fuure. Should he environmen goes hrough a cyclical change and becomes similar o he ime when he classifier in quesion was generaed, is error will hen be less han ½, and i will conribue o he curren decision of he ensemble. In order o combine he classifiers, however, we need one weigh for each, even hough he algorihm mainains a se of such error measures ε k for each classifier k=,,. The error measures are herefore combined hrough a weighed averaging (sep ). The weighing is done hrough a sigmoid funcion (Equaion 8), which gives a large weigh o errors on mos recen environmens, and less o errors on older environmens. We emphasize, however, ha his process does no give less weigh o old classifiers. I is heir error on old environmens ha is weighed less. Therefore, a classifier generaed long ime ago can in fac receive a large voing weigh, if is error on he recen environmens is low. The errors so weighed as described above are hen combined o obain a weighed average error (Eq. 9). The logarihm of he inverse of his weighed error average hen consiues he final voing weigh W k for classifier k a ime insan (Sep 6, Eq. ). Finally all classifiers are combined hrough weighed majoriy voing (Sep 7, Eq. ). 3 Experimens and Resuls Two experimens were designed o es he abiliy of he algorihm o rack a nonsaionary environmen. In boh experimens, daa was drawn from Gaussian disribuions so ha he opimal Bayes error could be compued and compared a each ime sep agains he Learn ++.NSE ensemble as well as single classifier performances. Naive Bayes was used as he base classifier in all experimens. In he firs experimen, concepually illusraed in Fig. 2, he disribuions for wo of he hree classes were moved in a concep drif scenario, where boh he means and he variances of he disribuions were changed. In fac, he disribuion of one class has compleely replaced he oher. The enire change was compleed in ime seps, however, in each case, we allowed he algorihm o see very lile of he environmen: a mere 2 samples per class. The disribuion for he hird class lef unchanged. We hen racked he performance of he Learn ++.NSE ensemble, a single classifier, and he Bayes classifier, a each ime sep. Figure 3 shows four independen rials of he percen classificaion performances on he enire feaure space, calculaed wih

6 An Ensemble Approach for Incremenal Learning in Nonsaionary Environmens 49 y 8 Class [ 8 3.] Class 3 (no change) [ ] Class new [8 ] Class 2 [8 4 ] 2 [ μ x μ y σ x σ y ] Pah of drif [8 2 2] Class 2 new 8 x Fig. 2. Concep drif simulaion 3-class experimen. Classes and 2 move and change variance. % performance on he enire feaure space Learn ++.NSE Single Classifier Bayes Classifier Time sep Fig. 3. Comparaive performances on four independen single rials respec o individual insances probabiliy of occurrence (o preven insances away from decision boundaries arificially increasing he performance). Several ineresing observaions can be made. Firs, all four performance rends indicae ha he Learn ++.NSE racks he Bayes classifier very closely, and as expeced, i has smaller performance variance han he single classifier.

7 496 M.D. Muhlbaier and R. Polikar Second, he ensemble performance is, in general, beer han he single classifier, and more closely follows he Bayes classifier. This is imporan since a single Naïve Bayes classifier esed on he mos recen daa on which i was rained would normally be expeced o do very well. The ensemble is using is combined knowledge from all raining sessions o correcly idenify hose areas ha have changed due o concep drif as well as hose ha have no changed. Third, he performance gap beween he ensemble and he single classifier appear o be widening in ime, in favour of he ensemble. Tha is, he performances of single classifier and Learn ++.NSE sar similar, bu in ime he ensemble increasingly ouperforms he single classifier despie occasional good single classifier performances. Since he large variance in he single classifier performance makes i difficul o deermine he validiy of such a claim, we looked a he average performance of independen rials of he same experimen. Performance resuls in Figure 4 indicae ha he Learn ++.NSE ensemble increasingly and significanly ouperforms he single classifier. Finally, noe ha wha appears o look like an overall performance decline (from 9% o 7%-8% range) in Fig.2 and Fig. 4 is irrelevan for our purposes. Such a decline merely indicaes ha he underlying classificaion problem is geing increasingly difficul in ime (see Fig. 2), as evidenced by he declining opimal Bayes classifier performance. We are primarily concerned wih how well Learn ++.NSE racks Bayes classifier, as we canno expec any algorihm o do any beer. % performance on he enire feaure space Learn ++.NSE Singe Classifier Bayes Classifier Time sep Fig. 4. Comparaive performances - average of independen rials We have also designed a second, four-class experimen, he non-saionary environmen of which was subsanially more challenging, including classes swiching places, changing variances, drifing in and ou of heir original disribuions in a cyclical manner. Figure provides snapshos of he disribuions of four classes c ~ c 4 a =, =T/3, =2T/3 and =T, where T indicaes he ime of he las environmen updae. During = o =T/3, he variances of he class disribuions were modified; hen he means of he class disribuions drifed unil =2T/3, and finally boh he variances and means were changed during he las hird secion of he simulaion. There were a oal of 2 ime seps from = o =T, during which he disribuions briefly reurned o he viciniy of heir original neighbourhoods wice, before drifing again in oher direcions. Hence, he design simulaed a semi-cyclical non-saionary behaviour.

8 An Ensemble Approach for Incremenal Learning in Nonsaionary Environmens ime = C C ime = T/3 C C 3 C 2 C 3 C 2 C 4.4 ime = 2T/3.4 ime = T.2 C C 2 C 3 C 4.2 C C 3 C 2 C 4 Fig.. The snapshos of he drifing disribuions a four ime seps % Classificaion performance on he enire feaure space Variances of he class disribuions drif Learn++.NS Single Classifier Bayes Classifier Class disribuions drif from one locaion o anoher Boh class locaion drif and variance drif Time sep each indicaing a differen environmen Fig. 6. Classificaion performance resuls on he second experimen

9 498 M.D. Muhlbaier and R. Polikar The classificaion performance resuls of Learn ++.NSE, single classifier and he Bayes classifier are shown in Figure 6, where all performances are averages of independen rials. Once again, he ensemble was able o rack he Bayes classifier much closer han he single classifier, despie he convolued changes in he environmen. The ensemble performance was also significanly beer han he single Naïve Bayes classifier a he 9% level for all > seps. 4 Conclusions and Discussions We described an ensemble based algorihm for nonsaionary environmens. The algorihm creaes a single classifier for each daase ha becomes available, and keeps a record of he performance of each classifier on all environmens hroughou he raining. The classifiers are hen combined hrough a modified dynamically weighed majoriy voing, where he voing weighs are deermined as a measure of each classifier s performance on he curren environmen, weighed along wih he performances on previous environmens. All classifiers are reained, which allows he previously generaed classifiers o make significan conribuions o he ensemble decision, if such classifiers provide relevan informaion for he curren environmen. The algorihm has many characerisics ha are deemed desirable for online learners [3]: (i) he algorihm learns from only single pass of each daase, wihou revisiing hem. This propery of he algorihm is inheried from is predecessor Learn ++, as i was designed o learn incremenally wihou requiring access o previously seen daa; (ii) he algorihm has a relaively small compuaional complexiy ha is linear in he number of daases, or even possibly in he daa size, depending on he base classifier (since i can be used wih any supervised classifier); (iii) he algorihm possesses any-ime-learning capabiliy, i.e., if he algorihm is sopped a ime, he algorihm provides he curren bes represenaion of he environmen a ha ime. As menioned earlier in our jusificaion for anoher ensemble based nonsaionary learning algorihm, he success of any algorihm depends much on how well is characerisics mach hose of he problem. I is herefore appropriae, and in fac necessary, o esablish wha such characerisics are for Learn ++.NSE. Specifically, when would we expec his algorihm o do well? The srucure of Learn ++.NSE makes i paricularly useful if he nonsaionary environmen provides a sequence of relaively small daa, ha by iself is no sufficien o adequaely represen he curren sae of he environmen. Then, a single classifier generaed wih such daa would no be able o appropriaely characerize he decision boundary. Only a subse of classes being represened in each daase is anoher scenario of nonsaionary learning, and he performance of Learn ++.NSE on such scenarios is par of our planned fuure work. The algorihm described here is cerainly no fully developed ye, and much work needs o be done. The algorihm, while inuiive, is based on heurisic ideas, and here are no heoreical performance guaranees. This is in par because we have no placed any resricions on he environmen. However, a careful analysis of he algorihm is necessary o deermine how i reacs o differen scenarios of nonsaionary environmens, and specifically o differen raes of change. Of paricular ineres is how well he algorihm would be able o rack he nonsaionary environmen, if he environmen changed faser, say for example, i made he same oal change in T/2 or T/4 ime seps raher han in T seps?

10 An Ensemble Approach for Incremenal Learning in Nonsaionary Environmens 499 While he algorihm is a is early sages of developmen, is iniial performance has been promising, moivaing us for furher opimizaion and developmen. Fuure effors will include he above menioned analysis for racking / esimaing he environmen s rae of change, and he algorihm s response o such change. Acknowledgmens. This maerial is based upon work suppored by he Naional Science Foundaion under Gran No. ECS References [] Schlimmer, J. C. and Granger, R. H.; Incremenal Learning from Noisy Daa. Machine Learning (986) [2] Helmbold, D. P. and Long, P. M.; Tracking drifing conceps by minimizing disagreemens. Machine Learning 4 (994) [3] Widmer, G. and Kuba, M.; Learning in he presence of concep drif and hidden conexs. Machine Learning 23 (996) 69-. [4] Case, J., Jain, S., Kaufmann, S., Sharma, A., and Sephan, F.; Predicive learning models for concep drif. Theoreical Compuer Science 268 (2) [] Liao, Y., Vemuri, V. R., and Pasos, A.; Adapive anomaly deecion wih evolving connecionis sysems. Journal of Nework and Compuer Applicaions 3 (27) 6-8. [6] Casillo, G., Gama, J., and Medas, P.; Adapaion o Drifing Conceps. Progress in Arificial Inelligence, Lecure Noes in Compuer Science 292 (23) [7] Gama, J., Medas, P., Casillo, G., and Rodrigues, P.; Learning wih Drif Deecion. Advances in Arificial Inelligence - SBIA 24, Lecure Noes in Compuer Science 37 (24) [8] Cohen, L., Avrahami-Bakish, G., Las, M., Kandel, A., and Kiperszok, O.; Real-ime daa mining of non-saionary daa sreams from sensor neworks. Informaion Fusion In Press, (27). [9] Maloof, M. A. and Michalski, R. S.; Incremenal learning wih parial insance memory. Arificial Inelligence 4 (24) [] Black, M. and Hickey, R.; Learning classificaion rules for elecom cusomer call daa under concep drif. Sof Compuing - A Fusion of Foundaions, Mehodologies and Applicaions 8 (23) 2-8. [] Rukowski, L.; Adapive probabilisic neural neworks for paern classificaion in imevarying environmen. IEEE Transacions on Neural Neworks (24) [2] Cheng-Kui, G., Zheng-Ou, W., and Ya-Ming, S.; Encoding a priori informaion in neural neworks o improve is modeling performance under non-saionary environmen. Inernaional Conference on Machine Learning and Cyberneics (ICMLC 24) (24) [3] Kuncheva, L. I.; Classifier Ensembles for Changing Environmens. Muliple Classifier Sysems (MCS 24), Lecure Noes in Compuer Science 377 (24) -. [4] Blum, A.; Empirical Suppor for Winnow and Weighed-Majoriy Algorihms: Resuls on a Calendar Scheduling Domain. Machine Learning 26 (997) -23. [] Oza, N.; Online Ensemble Learning, Ph.D. Disseraion, (2) Universiy of California, Berkeley. [6] Kyosuke, N., Koichiro, Y., and Takashi, O.; ACE: Adapive Classifiers-Ensemble Sysem for Concep-Drifing Environmens. Muliple Classifier Sysems (MCS 2), Lecure Noes in Compuer Science 34 (2) 76-8.

11 M.D. Muhlbaier and R. Polikar [7] Sree, W. N. and Kim, Y.; A sreaming ensemble algorihm (SEA) for large-scale classificaion. Sevenh ACM SIGKDD Inernaional Conference on Knowledge Discovery & Daa Mining (KDD-), (2) [8] Chu, F. and Zaniolo, C.; Fas and Ligh Boosing for Adapive Mining of Daa Sreams. Advances in Knowledge Discovery and Daa Mining, Lecure Noes in Compuer Science 36 (24) [9] Polikar, R., Upda, L., Upda, S. S., and Honavar, V.; Learn++: an incremenal learning algorihm for supervised neural neworks. Sysems, Man and Cyberneics, Par C, IEEE Transacions on 3 (2)