Suppo Veco Machine CSL 3 ARIFICIAL INELLIGENCE SPRING 4
Suppo Veco Machine O, Kenel Machine Diciminan-baed mehod olean cla boundaie Suppo veco coni of eample cloe o bounday Kenel compue imilaiy beeen eample omap inance pace o a highe-dimenional pace hee (hopefully) linea model uffice Chooing he igh kenel i cucial Kenel machine among be-pefoming leane CSL 3 ARIFICIAL INELLIGENCE, INDIAN INSIUE OF ECHNOLOGY ROPAR
Suppo Veco Machine O = uppo veco magin CSL 3 ARIFICIAL INELLIGENCE, INDIAN INSIUE OF ECHNOLOGY ROPAR 3
Opimal Sepaaing Hypeplane X, find and hee uch ha if if C C fo hich can be fo eien a Noe e an +, no Wan inance ome diance fom hypeplane CSL 3 ARIFICIAL INELLIGENCE, INDIAN INSIUE OF ECHNOLOGY ROPAR 4
Magin Diance fom inance o hypeplane + ( ) o Diance fom hypeplane o cloe inance i he magin magin CSL 3 ARIFICIAL INELLIGENCE, INDIAN INSIUE OF ECHNOLOGY ROPAR 5
Opimal Sepaaing Hypeplane min ubjec o, Quadaic opimizaion poblem ih compleiy polynomial in d (#feaue) Kenel ill evenually map d-dimenional pace o highe-dimenional pace Pefe compleiy no baed on #dimenion CSL 3 ARIFICIAL INELLIGENCE, INDIAN INSIUE OF ECHNOLOGY ROPAR 7
Opimal Sepaaing Hypeplane Conve opimizaion poblem o depend on numbe of aining eample N (no d) osill polynomial in N Bu opimizaion ill depend only on cloe eample (uppo veco) oypically N CSL 3 ARIFICIAL INELLIGENCE, INDIAN INSIUE OF ECHNOLOGY ROPAR 8
Lagange Muliplie Reie quadaic opimizaion poblem uing Lagange muliplie α, N CSL 3 ARIFICIAL INELLIGENCE, INDIAN INSIUE OF ECHNOLOGY ROPAR 9 N N N p L, ubjec o min
Lagange Muliplie Equivalenly, e can maimize L p ubjec o he conain: Plugging hee ino L p CSL 3 ARIFICIAL INELLIGENCE, INDIAN INSIUE OF ECHNOLOGY ROPAR N p N p L L
Lagange Muliplie Maimize L d ih epec o α only CSL 3 ARIFICIAL INELLIGENCE, INDIAN INSIUE OF ECHNOLOGY ROPAR and o ubjec L d,
Suppo Veco Machine Mo α = oi.e., ( + ) > ( lie ouide magin) Suppo veco: uch ha α > oi.e., ( + ) = ( lie on magin) = Σ α = fo any uppo veco oypically aveage ove all uppo veco Reuling diciminan i called he uppo veco machine (SVM) CSL 3 ARIFICIAL INELLIGENCE, INDIAN INSIUE OF ECHNOLOGY ROPAR
Sof Eo magin O = uppo veco CSL 3 ARIFICIAL INELLIGENCE, INDIAN INSIUE OF ECHNOLOGY ROPAR 3
Sof Magin Hypeplane Daa no linealy epaable Find hypeplane ih lea eo Define lack vaiable ξ oing deviaion fom he magin CSL 3 ARIFICIAL INELLIGENCE, INDIAN INSIUE OF ECHNOLOGY ROPAR 4
Sof Eo Coecly claified eample fa fom magin (ξ = ) Coecly claified eample, bu inide he magin ( < ξ < ) Incoecly claified eample (ξ ) Sof eo = CSL 3 ARIFICIAL INELLIGENCE, INDIAN INSIUE OF ECHNOLOGY ROPAR 5
Sof Magin Hypeplane Lagangian equaion ih lack vaiable C i penaly faco μ, ne e of Lagange muliplie Wan o minimize L p CSL 3 ARIFICIAL INELLIGENCE, INDIAN INSIUE OF ECHNOLOGY ROPAR 6 p C L
Sof Magin Hypeplane L d ubjec o C i a egulaizaion paamee ohigh C high penaly fo non-epaable eample (ovefi) olo C le penaly (undefi) odeemine uing validaion e (C= ypical) and C, CSL 3 ARIFICIAL INELLIGENCE, INDIAN INSIUE OF ECHNOLOGY ROPAR 9
Kenel o ue peviou appoache, daa mu be nea linealy epaable If no, pehap a anfomaion φ() ill help φ() ae bai funcion φ CSL 3 ARIFICIAL INELLIGENCE, INDIAN INSIUE OF ECHNOLOGY ROPAR
Kenel anfom d-dimenional pace o k- dimenional z pace uing bai funcion φ() z=φ() hee z j = φ j (), j=,,k Inead of, aume z = φ () CSL 3 ARIFICIAL INELLIGENCE, INDIAN INSIUE OF ECHNOLOGY ROPAR k j j j g g ) ( ) ( ) ( ) ( φ z z
Sof Magin ih Kenel Replace inne poduc of bai funcion φ( ) φ( ) ih kenel funcion K(, ) CSL 3 ARIFICIAL INELLIGENCE, INDIAN INSIUE OF ECHNOLOGY ROPAR p C L ) ( φ, and ubjec o ) ( C L d φ φ d K L ), (
Kenel Funcion Kenel K(, ) compue z-pace poduc φ( ) φ( ) in -pace Mai of kenel value K, hee K = K(, ), called he Gam mai K hould be ymmeic and poiive emidefinie CSL 3 ARIFICIAL INELLIGENCE, INDIAN INSIUE OF ECHNOLOGY ROPAR 3 K g g φ φ φ φ z,
Kenel Funcion Polynomial kenel of degee q If q=, hen ue oiginal feaue Fo eample, hen q= and d= CSL 3 ARIFICIAL INELLIGENCE, INDIAN INSIUE OF ECHNOLOGY ROPAR 4 q K, y y y y y y y y K,,,,,, y y
Kenel Funcion Polynomial kenel of degee magin O = uppo veco CSL 3 ARIFICIAL INELLIGENCE, INDIAN INSIUE OF ECHNOLOGY ROPAR 5
Kenel Funcion Radial bai funcion (Gauian kenel) K, ep i he cene i he adiu Lage implie moohe boundaie CSL 3 ARIFICIAL INELLIGENCE, INDIAN INSIUE OF ECHNOLOGY ROPAR 6
Kenel Funcion: Radial Bai Funcion CSL 3 ARIFICIAL INELLIGENCE, INDIAN INSIUE OF ECHNOLOGY ROPAR 7
K> Clae Lean K diffeen kenel machine g i () oeach ue one cla a poiive, emaining clae a negaive ochooe cla i uch ha i=agma j g j () obe appoach in pacice CSL 3 ARIFICIAL INELLIGENCE, INDIAN INSIUE OF ECHNOLOGY ROPAR 3
K> Clae Lean K(K-)/ kenel machine oeach ue one cla a poiive and anohe cla a negaive oeaie (fae) leaning pe kenel machine CSL 3 ARIFICIAL INELLIGENCE, INDIAN INSIUE OF ECHNOLOGY ROPAR 3
Summay: Kenel Machine Seek opimal epaaing hypeplane Suppo veco machine (SVM) find hypeplane uing only cloe eample Kenel funcion allo SVM o opeae in highe dimenion Chooing coec kenel i cucial Kenel machine among be-pefoming leane CSL 3 ARIFICIAL INELLIGENCE, INDIAN INSIUE OF ECHNOLOGY ROPAR 33