Relationship between the number of labeled samples and classification accuracy based on sparse representation

Transcription

1 Relationship between the nuber of labeled saples and lassifiation auray based on sparse representation 1 Shool of Coputer Siene and Engineering, Beifang University for Nationalities,Yinhuan, 75001,China Chunei Zhang Shool of Coputer Siene and Engineering,Beifang University for Nationalities,Yinhuan, 75001,China E-ail: hunei66@hotail.o Yunbin Zhang 3 Shool of Coputer Siene and Engineering, Beifang University for Nationalities, Yinhuan, 75001,China Sparse deoposition algorith an be ipleented to lassify high diensional data without diension redution. Epirial exainations have shown that the lassifiation auray of the hyperspetral data an be signifiantly iproved based on sparse representation fraework. However, the lassifiation auray of hyperspetral data is redued greatly if the quantity of saples is too sall. This paper represents an attept to fill the gap represented by the absene of the lower bound of labeled saple quantity of sparse representations in supervised lassifiation task, in order to fous future analyses towards the ost relevant aspets of iage lassifiation auray and data labeling ost. Based on the boundary onditions of the unique solution of sparse representation, the relationship between the lassifiation auray of hyperspetral data and the quantity of labeled saples is analyzed theoretially in this paper. The orrelation urves of the saple quantity and the lassifiation auray is figured with the Kennedy Spae Center (KSC) hyperspetral data. PoS(CENet015)014 CENet Septeber 015 Shanghai, China 1 Speaker Corresponding Author 3 This work is supported by National Natural Siene Foundation of China ( ) and Innovation Projets for graduate students of Beifang University for Nationalities. Copyright owned by the author(s) under the ters of the Creative Coons Attribution-NonCoerial-NoDerivatives 4.0 International Liense (CC BY-NC-ND 4.0).

2 Relationship between the nuber of labeled saples 1. Introdution The paradox of high diension of hyperspetral data is that, in one hand, any lassifiation ethods are not appropriate for dealing with high diensional data, so diension redution beoes a ain approah. On the other hand, high diension eans high inforation. If diension redution ethods are ipleented, the valuable inforation will be lost. Many lassifiation algoriths have been ipleented to lassify hyperspetral data. Support Vetor Mahine (SVM) is a lassial algorith, but it is sensitive to the kernel funtion. HSVM (Hierarhial SVM) is the iproveent of the SVM whih obine the Max-ut tree and SVM together in order to enhane the lassifiation auray [1]. Ditionary learning of sparse representation an obtain better lassifiation auray []. However, ditionary learning is tie onsuing. Sparse deoposition algorith is a fast and effetive fraework and an be ipleented to lassify high diensional data. The fraework has the advantage that it does not need diension redution, odel seletion and ditionary training. Eah test saple an be sparsely represented with only a few non-zero deoposition oeffiients by a few training saples of an over-oplete ditionary whih are onstruted by training saples diretly. Aording to the approxiation error, test saples an be ategorized to the orresponding lasses. Several researhers have iproved the lassifiation auray of hyperspetral data based on sparse representation fraework [3-5]. Epirial exainations have shown that the auray of hyperspetral lassifiation is redued greatly if the quantity of labeled saples is too sall. However, identifying ore labeled saples will ake the ost rise. To get the lower bound of the saple quantity and analyze the relationship between the quantity of labeled saples and lassifiation auray, we dedue a forula whih is the relationship between the residual error of sparse representation and the nuber of ditionary atos. Then through epirial data, the orrelation urve between the nuber of labeled saples and lassifiation auray is figured. The rest of paper is organized as follows. In the seond setion, we introdue the theoretial analysis of the relationship between the nuber of labeled saples and lassifiation auray. In the third setion, we analyze and show the experiental results. Finally, the fourth setion onludes this paper..theoretial Analysis In this setion, we introdue the theoretial analysis of the relationship between the nuber of labeled saples and lassifiation auray. PoS(CENet015)014.1 Sparse Representation Algorith We an onstrut a set of ditionaries aording to different ategories. Eah ditionary gets the labeled saples of the sae ategory fro hyperspetral data to for the overoplete ditionary. And let D =[d 1, d,, d M ](D ϵ R N*M ) denotes the ditionary of th ategory. Aording to the theory of sparse representation, eah test saple an be sparsely represented by a few training saples of an over-oplete ditionary with only a few non-zero deoposition oeffiients. We an get sparse deoposition oeffiient aording to forula (.1). aˆ( f ) = arg in f - Da + l a (.1) a 1 T a a1 a a M The above forula is an l 1 -regularized least squares proble. Where [,, ] = L is the oeffiient of sparse representation. Regularization paraeter λ is used to balane the data

3 Relationship between the nuber of labeled saples reonstrution error and sparsity. The threshold of paraeter λ is seleted aording to epirial value. Then the test saple f is ategorized to the orresponding lasses aording to forula (.). ( ) in ( ˆ ) in R f = f - D a (.) Where R ( f ) is the residual to reonstrut f by training saples in the ategory. The saller the value R ( f ) is, the higher the possibility of f belongs to the ategory of th.. Coherent The oherent oeffiient μ is a harater of ditionary. Its value equals to the axiu of inner produt between two distint atos in the ditionary [6]. μ := ax i j <d i, d j > s.t. i, j ϵ [1,,M] (.3) Generally, 0< μ <1. If µ has a very sall value, we say that the redundant ditionary is inoherent, and obviously, the oherent oeffiient of an orthogonal basis equals zero. When μ=1, then the ditionary ontains at least two idential atos [7]. Furtherore, in finite diension spae of N, the oherene of a onatenate ditionary of two orthonoral bases is >= 1 N [6]. For general redundant ditionary, a lower bound of oherene oeffiient is [8] Where M is the nuber of ditionary atos, N is the diensionality of data saple..3 Reovery and Convergene of MP Basis pursuit (BP),Mathing pursuit (MP) and its variants are the lassi algorith of sparse representation. For the Exat Reovery Condition of BP and OMP (Orthogonal athing pursuit), J.A.Tropp has given a unified suffiient ondition [4]: 1 1 < ( ) (.5). As long as whih is the nuber of nonzero entries of the oeffiients of sparse representation satisfies above forula (.5), OMP and BP both reover every superposition of atos fro D. In ost ases, the signal or iage an only be expressed by linear obination of soe atos in the atual ditionary. For MP algorith, the approxiation perforane an be desribed as follows [9]: Theore: Let{ f n } be a sequene of approxiants to f ϵ H where d in is the orresponding atos seleted through MP. Let >= (.4) 1 1 < ( ) (.6), 4 and f f = α d be a best approxiant to f fro D, i.e. i ϵ I i i f f f = σ ( f ) := inf { f D α, ard ( I ) <=, α ϵ I CI } (.7), then, there is an nuber N suh that the error after N steps satisfies M - N N( M -1) f - f <= 1+ 4s N (.8) PoS(CENet015)014 3

4 Relationship between the nuber of labeled saples during the first N steps MP piks up atos fro the best -ter approxiant: i n ϵ I ; if s < 3 s /, then 1 4 3s N <= +..ln 3 s 1 (.9).4 Monotoniity Analysis of Residual Error between Fro the forula (.6) and (.8), we an get a new inequality about the relationship fn - f and oherent oeffiient μ. f f - 1 N - < + (.10) Fro the forula (.4) and (.10), we an derive the following inequality between the nuber of ditionary atos and the residual error of sparse approxiate: f N - f < + N( M -1) M - N s (.11) The diension of ditionary is unhanged, so, in inequality (.11), N an be onsidered as a onstant. Then inequality (.11) an be written as So, there exist an λ ϵ (0,1]. variable M. N( N -1) fn - f < + N + M - N s fn (.1) - f will derease with the inrease of independent N( N -1) f - f = l + N + s M - N N By analyzing the onotoniity of equality (.13), we an find that variable fn (.13) - f will derease with the inrease of independent variable M. The lassifiation riterion of sparse representation ethod is easured by fn fn - f. In another word, the saller the residual error - f is, the higher the lassifiation auray is. When the nuber of atos of ditionary inreases, the residual error will dereases gradually, and the lassifiation auray will rise, so lassifiation auray is positively orrelated with the nuber of atos. Beause the atos of ditionary are labeled training saples, so the lassifiation auray is positively orrelated with the nuber of the labeled saples. PoS(CENet015) Experiental Analysis and Conlusion KSC hyperspetral data is used in our experient. The results of our experient advoate that the nuber of labeled saples is positively orrelated with lassifiation auray. The nuber of labeled saples plays a ritial role on lassified auray. 3.1 Data set The speifi inforation of hyperspetral data that eployed in experient inludes the following. 4

5 Relationship between the nuber of labeled saples The hyperspetral iage data was aquired by the National Aeronautis and Spae Adinistration (NASA) Airborne Visible/Infrared Iaging Spetroeter (AVIRIS) instruent. AVIRIS aquired iage data over the Kennedy Spae Center (KSC), Florida, on Marh 3, The data has 4 bands and the enter wavelengths is fro 400~500n. After reoving water absorption and low signal-to-noise (SNR) bands, 176 bands were seleted for the analysis. The hyperspetral data ontains 13 typial lasses of land over and 511 saple points. The detail of the data set is tabulated in table 1. Table 1: The Inforation of KSC Data Class Class nae No.saples 1 Srub 761(14.6%) Willow swap 43(4.66%) 3 Cabbage pal hank 56(4.9%) 4 Cabbage pal/oak hank 5(4.84%) 5 Slash pine 161(3.07%) 6 Oak/broadleaf haok 9(4.38%) 7 Hardwood swap 105(.0%) 8 Grainoid arsh 431(8.7%) 9 Spartina arsh 50(9.99%) 10 Cattail arsh 404(7.76%) 11 Salt arsh 419(8.04%) 1 Mud flats 503(9.66%) 13 Water 97(17.8%) 3. Data Preproessing and the Algorith Steps In the experient, firstly, 5% data saples are seleted randoly into the test set, the rest of 75% data are used as the pool of training exaples. Then nine different training sets with different proportion of saples seleted randoly fro the training exaple pool are investigated, e.g. 5%, 8%, 10%, 1%, 15%, 0%, 30%,50% and 75%. Eah training sets with orresponding proportion are exeuted 10 ties. Pixel values with respet to eah band are noralized. The experiental steps are as follows: Classifiation algorith Input: X: Training set (labeled exaple set) F: Test set (Unlabeled exaple set) Output: (y): the ategory of eah test saple f. Eah training saple x ϵ X or test saple f ϵ F is noralized. Use the training set to for the over-oplete ditionary D. Use lasso algorith to deopose the test saple sparsely to get the sparse representation oeffiient â. Aording to forula (.),alulate the residual error R ( f ) of test saple whih was sparse approxiation by ditionary oposed of saples of the th ategory. Then the test saple is ategorized to the orresponding ategory aording to the iniu residual error. Stop repeat step 3and 4 until all the test saples are ategorized. PoS(CENet015)014 5

6 Relationship between the nuber of labeled saples 3.3 Experient Results and Analysis The lassifiation auray is the average of the ten experiental results. The lassifiation ethod used for oparison is HSVM. Table opares the lassifiation perforanes of HSVM and sparse representation (SR) at diffident sapling rates. For the two opared algoriths on eah data set, the average lassifiation auray inrease as the size of labeled data set gets large and SR onsistently deonstrates better perforane opared to HSVM. Training rate Nuber of sapling HSVM SR 5% % % % % % % % % Table : The Classifiation Auraies of HSVM and SR Furtherore, table desribes the orresponding relationship between lassifiation auray and different rate of the training saple, naely, the nuber of training saple. Figure 1 draws the relation urve between lassifiation auray and the nuber of training saple aording to the speifi experient results. PoS(CENet015)014 Figure 1: the relation urve between lassifiation auray and the nuber of training saple Through analyzing table and figure 1, we an easily find that the lassifiation auray onotonially inrease with the nuber of labeled saples. In the experient of SR, the proportions of seleted training saples are gradually inreased fro 5% to 75%. The lassifiation auray is gradually inreased fro % to %. 4. Conlusion In this paper, we proved that lassifiation auray of hyperspetral data is positively orrelated with the nuber of labeled saples. Aording to the theory of sparse representation, firstly, we derive the relationship between the nuber of labeled saples and lassifiation auray. Then through the experient, we give the orrelation urves between the nuber of labeled saples and lassifiation auray. When the nuber of labeled saples reahes 10% 6

7 Relationship between the nuber of labeled saples or ore, the lassified auray an be ensured ore than 90%. Obviously, the nuber of labeled saples plays a ritial role on lassifiation auray. Referenes [1] Y.C. Chen, M. M. Crawford, J. Ghosh. Integrating support vetor ahines in a hierarhial output spae deoposition fraework[c]. Geosiene and Reote Sensing Syposiu, IEEE International. Pisataway,N.J. pp,949-95(004) [] J. Mairal, F. Bah, J. Prone, G. Sapiro. Online Learning for Matrix Fatorization and Sparse Coding[J]. Journal of Mahine Learning Researh.11(1):19-60(010) [3] M.S. Cui, S. Prasad. Class-Dependent Sparse Representation Classifier for Robust Hyperspetral Iage Classifiation[J]. Geosiene and Reote Sensing, IEEE Transations on.53(5): (015) [4] Y.F. Tang, X.M. Li, Y. Xu, Y. Liu, J.Z. Wang, C.Y. Liu. Hyperspetral iage lassifiation using sparse representation-based lassifier[c]. Geosiene and Reote Sensing Syposiu,IEEE International. Pisataway,N.J. pp, (014) [5] S. Qazi, L.X. Shi, L.M. Tao, S.Q. Yang. A l1-iniization Based Approah for Hyperspetral Data Classifiation[J]. Key Engineering Materials.500: (01) [6] J.A. Tropp. Greed is good: algorithi results for sparse approxiation[j]. Inforation Theory,IEEE Transations on.50(10):31-4 (004) [7] C.M. Zhang, Z.K. Yin, X.D. Chen, M.X. Xiao. Signal overoplete representation and sparse deoposition based on redundant ditionaries[j]. Chinese Siene Bulletin. 50(3): (005) [8] A.F. Abdelnour, I.W. Selesnik. Syetri nearly shift-invariant tight frae wavelets[j]. Signal Proessing, IEEE Transations on. 53(1): 31-39(005) [9] R. Gribonval, P. Vandergheynst. On the exponential onvergene of Mathing Pursuits in quasiinoherent ditionaries[j]. Inforation Theory,IEEE Transations on.5(1):55-61(006) PoS(CENet015)014 7