Proceedngs of the 6th WSEAS Internatona Conference on Apped Informatcs and Communcatons, Eounda, Greece, August 8-0, 006 (pp388-393) A Nove Herarchca Method for Dgta Sgna ype Cassfcaton AAOLLAH EBRAHIMZADEH, SEYED ALIREZA SEYEDIN Department of Eectrca and Computer Engneerng Nousrvan Insttute of echnoogy, Ferdows Unversty of Mashad IRAN Abstract: - Automatc sgnas type cassfcaton (ASC) s a technque for recognzng the moduaton scheme of an ntercepted sgna and has seen ncreasng demand n both mtary and cvan. Most of prevous technques can dentfy ony a few knds of sgna types. Aso, usuay, they request hgh SNR to acheve a mnmum acceptabe performance and don t ncude hgher order and new types of sgnas (e.g. QAM types).he work presented here proposes nove hgh performance method for dentfcaton of dgta sgna types. In ths method t s proposed a herarchca mutcass cassfer based on support vector machne (SVM). he nputs of ths cassfer are hgher order moments and cumuants (up to eght). Genetc agorthm (GA) s used to seect the best mode of SVMs n order to mprove the performance. Smuaton resuts show that proposed method can recognze a ot of dgta sgna types and acheves a hghy probabty of cassfcaton even at ow sgna to nose ratos (SNRs). Key-Words: - Pattern recognton, moduaton, machne earnng, support vector machne, mode seecton, eght order statstcs. Introducton Automatc sgnas type cassfcaton s a technque for recognzng the moduaton scheme of an ntercepted sgna. It pays an mportant roe for varous appcatons ad purposes. For exampe, n mtary appcaton, moduaton cassfcaton can be empoyed for eectronc surveance, nterference dentfcaton, montorng; n cv appcatons, t can be used for spectrum management, network traffc admnstraton, dfferent data rate aocaton, sgna confrmaton, nterference dentfcaton, software rados, mutdrop networks, ntegent modem, etc. In the past, sgna type recognton reed mosty on operators scannng the rado frequency spectrum wth a wde-band recever and checkng t vsuay on some sort of dspay. Ceary, these methods reed very much on the operators' sks and abtes. hese mtatons then ed to the deveopment of more automated moduaton recognzers. One semautomatc approach was to run the receved sgna through a number of demoduators and then have an operator determne the moduaton format by stenng to the output of each demoduator. hs approach s however not very practca anymore due to the new dgta technques that transfer both voce and data. hen technques for automatc sgna type cassfcaton (ASC) started to emerge. Whst eary researches ASC were concentrated on anaogue moduatons and ower orders of some types of dgta moduatons, such as frequency shft keyng (FSK), amptude shft keyng (ASK), the recent contrbutons n the subect focus more on hgher order and new types of dgta sgnas ke phase shft keyng (PSK), amptude shft keyng (QAM). Prmary, ths s due to ncreasng usage of such moduatons n many nove appcatons. ASC technques usuay can be categorzed n two man prncpes: the decson theoretc (D) and the pattern recognton (PR). D approaches use probabstc and hypothess testng arguments to formuate the recognton probem and to obtan the cassfcaton rue [-3]. he maor drawbacks of these approaches are ther very hgh computatona compexty, dffcuty wthn the mpementaton of decson rue, and ack of robustness to mode msmatch. Aso, these methods have dffcutes to set the correct threshod vaues. PR approaches, however, do not need such carefu treatment. hese methods are smpe to mpement.hey can be further dvded nto two subsystems: the feature extracton subsystem and the cassfer subsystem. he former extracts promnent characterstcs from the raw data, whch are caed features, and the cassfer dentfes the type of sgna based on the extracted features [4-]. It s showed that n moduaton dentfcaton area, ANNs especay mutayer perceptron (MLP) outperform other cassfers [8-]. However, wth regard to effectveness of ANNs, there are some probems worth mentonng. he tradtona ANNs have mtatons on generazaton gvng rse to modes that can overft to the tranng data. ANNs show
Proceedngs of the 6th WSEAS Internatona Conference on Apped Informatcs and Communcatons, Eounda, Greece, August 8-0, 006 (pp388-393) poor performances n ow SNRs and hgher number of moduaton schemes usuay requres a more compex neura network and hgher tranng tme [3, 4].hs defcences are due to the optmzaton agorthms used n ANNs for seecton of parameters and the statstca measures used to seect the mode. Recenty, support vector machnes (SVMs), based on statstca earnng theory are ganng appcatons n the area of machne earnng and pattern recognton because of hgh accuracy and exceent generazaton capabty. In [3, 4], the authors showed that usng SVM n content of sgna type dentfcaton acheves hghy success rate. hus n ths paper we use SVM as a cassfer. Seecton the proper mode of cassfer can mprove a substanta amount of s performance. In ths paper we have used the GA for SVM s mode seecton. Penty of ASC methods are abe to recognze ony a few knds of sgnas types and/or ower order of moduatons; aso, usuay, they request hgh SNR for havng a mnmum acceptabe performance. hs s due to facts: cassfer and features. Athough the cassfer has an mportant roe n cassfcaton, t shoud be mentoned that the features have vta roe. In ths work we have used hgher order moments and cumuants as features. he work presented here, proposes a hgh effcent ASC method, whch s abe to recognze dfferent type of receved sgnas. he paper organzed as foows. Secton presents feature extracton. Aso the consdered moduaton set s ntroduced n ths secton. Secton 3, descrbes the cassfer. Secton 4, ntroduced the GA that s used for mode seecton of SVMs. Secton 5, shows some smuaton studes. Fnay, n secton 6 concusons are presented. Feature extracton In moduaton dentfcaton probem, fndng the proper features s very mportant. For exampe QAM moduaton schemes contan nformaton n both amptude and phase (that are regarded as compex sgnas), thus fndng the proper feature that coud be abe to dentfy them (especay n case of hgher order and/or non-square) s dffcut. Based on our researches, the hgher order moments and hgher order cumuants up to eghth acheve the most hghy performances to dscrmnatng of dgta moduatons such as consdered set n ths paper. hese features have many advantages e.g. they provde a good way to descrbe the shape of the probabty densty functon. Probabty dstrbuton moments are a generazaton of concept of the expected vaue [3]. he auto-moment of the random varabe may be defned as foows: p q q M pq = E[ s ( s ) ] () where p caed moment order. Assume a zero-mean dscrete based-band sgna sequence of the form s k = ak + bk. Usng the (5), the expressons for dfferent order may be easy derved. th he symbosm for p order of cumuant s defned as: C pq = Cum[ 3 s,..., s, s 443,..., s ] () ( p q) terms ( q) terms here are reatons between moments and cumuants. he n th order cumuant s a functon of the moments of orders up to ncudng n. Moments s expressed n terms of cumuants as: M [ s ] { } { },.., sn = Cum s... um s v (3) v v where the summaton ndex s over a parttons v = v,..., v ) for the set of ndexes (,,..., n ), and q ( q s the number of eements n a gven partton. Cumuants may be aso be derved n terms of moments. he n order cumuant of a dscrete sgna th s(n) s gven by: q Cum [ s,.., s ] = ( ) ( )! [ ].. [ ] q E n s E s (4) v v v q In ths paper the consdered moduatons set s {ASK4, ASK8, PSK, PSK4, PSK8, Star-QAM8, V9, QAM6, QAM64, and QAM8}. We have computed the features of consdered dgta moduaton set. 3 Cassfer As sad we have used a herarchca SVM-based structure as cassfer. Support Vector Machne (SVM) s a supervsed machne earnng technque that can be apped as robust too for bnary and mutcass cassfcatons [5]. In case of SVMs, structura rsk mnmzaton (SRM) prncpe s used mnmzng an upper bound on the expected rsk whereas n ANNs, emprca rsk mnmzaton (ERM) s used mnmzng the error on the tranng data. he dfference n RM eads to better generazaton performance for SVMs than ANNs. he SVM performs cassfcaton tasks by constructng optma separatng hyperpanes (OSH). he dea of bnary SVM can be expressed as foows [6]. 3. Bnary SVM Suppose the tranng set, d ( x, y ), =,,...,, x R, y {, + } can be separated by the hyperpane w x + b = 0, where w r s
Proceedngs of the 6th WSEAS Internatona Conference on Apped Informatcs and Communcatons, Eounda, Greece, August 8-0, 006 (pp388-393) weght vector and b s bas. If ths hyperpane maxmze the margn, then the foowng nequaty s vad for a nput data: y ( w x,for a x =,,..., (5) he margn of the hyper-pane s / w r, thus, the probem s: maxmzng the margn by mnmzng of w subect to constrants (5), that s a convex quadratc programmng (QP) probem. hs probem has a goba optmum, and Lagrange mutpers are used to sove t: LP = w α [ y ( w x ] (6) = where a, =,..., are the Lagrange mutpers ( α 0).he souton to ths QP probem s gven by mnmzng L P wth respect to w and b. After dfferentatng L P wth respect to w and b and settng the dervatves equa to 0, yeds: w = α x (7) = y It can obtan the dua varabes Lagrangan by mposng the Karush-Kuhn-ucker (KK) condtons: L d = = α w = α αα y y x x = = = o fnd the OSH, t must maxmze (8) L d under the constrants of α = 0,andα 0. Note that the = y Lagrange mutpers are ony non-zero ( α f 0 ) when y ( w x =. hose tranng ponts for whch the equaty n (5) hods are caed support vectors (SV) that can satsfyα f 0. he optma weghts are gven by (7) and the bas s gven by: b = y w x (9) for any support vector x. he optma decson functon (ODF) s then gven by: f ( x) = sgn( y α x x + b ) (0) = where α s are optma Lagrange mutpers. For nput data wth a hgh nose eve, SVM uses soft margns can be expressed as foows wth the ntroducton of the non-negatve sack varabesξ, =,..., : y ( w x ξ for =,,..., () o obtan the OSH, t shoud be mnmzng the k Φ = w + C ξ subect to constrants (), = where C s the penaty parameter, whch contros the tradeoff between the compexty of the decson functon and the number of tranng exampes mscassfed,.e., contros the tradeoff between margn maxmzaton and error mnmzaton. hs probem can be soved wth smar method. In the nonneary separabe cases, the SVM map the tranng ponts, nonneary, to a hghdmensona feature space usng kerne functon K( x r r, x ), where near separaton may be possbe. he most famous kerne functons are near, poynoma, rada bass functon (RBF), and sgmod. For exampe RBF w be shown by: K( x, y) = exp( x y / σ ) () where σ s the wdth of the RBF kerne. After a kerne functon s seected, the QP probem s: Ld = α αα y y K( x, x ) (3) the = = = α s derved by: α = arg max L 0 α C; =,,..., ; α d = α y = 0 (4) After tranng, the foowng, the decson functon, becomes: f ( x) = sgn( y α K( x, x ) + b ) (5) = he performance of SVM can be controed through a few free parameters ke the term C and the kerne parameter whch caed are hyperparameters. 3. Mutcass SVM As sad SVM was orgnay deveoped for bnary cassfcaton probems, but t can be used for mutcass cassfcaton. here are severa approaches avaabe to extend bnary SVMs to mut-cass probems that fa nto two categores: the oneaganst-others method and one-aganst-one method [7]. In ths research, we have proposed a nove, smpe and effectve souton for mutcass SVMbased that s herarchca. In ths scheme at frst a of nputs fed to one SVM. hs SVM separates them n two groups. Each of these groups fed to another SVMs and process w be contnued t a of patterns are cassfed. he detas of ths structure ntroduced n secton 5.
Proceedngs of the 6th WSEAS Internatona Conference on Apped Informatcs and Communcatons, Eounda, Greece, August 8-0, 006 (pp388-393) 4 Mode seecton and GA Drvng the optma vaues of SVMs parameters are mportant to acheve hgh generazaton abty and performance. GAs wth ther characterstcs of hgh effcency and goba optmzaton are wdey apped n many areas. In ths paper, accordng the determned approprate ftness functon for GA operaton, an effectve strategy for parameters seecton for SVM s proposed by usng the GA. GA s a stochastc optmzaton agorthm whch adopts Darwn s theory of survva of the fttest. o appy GAs to the SVM parameters seecton probem, one has to consder the foowng ssues: the encodng scheme, the methodoogy to produce the nta popuaton, the ftness functon and the genetc operators such as reproducton, crossover and mutaton. 4. Encodng Scheme Here rea-encoded scheme s seected as the representaton of the parameters n ths paper. he research space of these parameters s C [5 : 5 : 50], σ [0.: 0.: 3.0] 4. Produce the Inta Popuaton Because the rea-coded scheme s used, the souton space concdes wth the chromosome space. Consderng the bgger popuaton w enarge the GA runnng tme and dsperse the conformaton of the popuaton, the sze of popuaton, pop_sze s choose 6 n orderng to avodng the convergence of the popuaton becomes dffcut. For producng the nta popuaton, the nta vaues of the desgned parameters are dstrbuted n the souton space as even as possbe. 4.3 Ftness Functon Accordng to the aforementoned anayss, the average performance of the SVM cassfer s depended on E { R / γ ) and not smpy on the arge margn γ. he Radus-margn bound [8] s proposed as the ftness functon R = (6) γ where γ denotes the margn, s the sze of the tranng sampes, R s the radus of the smaest sphere contanng the tranng data, R = 0.5. 4.4 Genetc Operators Genetc operator ncudes the foowng three basc operators such as seecton, crossover and mutaton. Here, a heurstc search strategy s adopted to reaze the genetc optmzaton for automatc parameters seecton. he adopted GA operators are brefy presented as foows. 4.4. Seecton operator Seecton operators s composed of a copy seecton operaton and a survve seecton operaton. Here the method of survva of the fttest was used to seect the next generaton ndvdua. Gven the ftness functon ft ( a ) of the ndvdua a, the probabty of a seected as the next generaton one s as foow: ft( a ) P( a ) = pop _ sze (7) pop _ sze ft( a ) = 4.4. Crossover operator he means of crossover mpement s cosey ntegrated to the encodng scheme. Due to the reaencodng scheme s utzed; the crossover operator n ths paper can be defned as [9]: P = ap + ( a) P (8) where P' s the offsprng after crossover operaton,p and P are two parents to be mpemented the crossover operaton, and a s a constant whch beongs to (0,).Here a =0.5. 4.4.3 Mutaton operator How the bgger vaue of the mutaton operator s chose to mantan the dversty of the popuaton n the eary GA operaton and avod the precocty? he adaptve mutaton probabty s adopted n ths paper to sove the above two probems as foows: exp( b t / ) P m = (9) pop _ sze L where t s the generaton of the genetc teraton, pop _ sze s the sze of the popuaton, L s the ength of the ndvdua, b=.5 s a preset parameter. 4.4.4 he stoppng crtera In genera GA agorthm, termnate the program when the best ftness has not changed more than a very sma vaue,.e. 0-6 over the ast generatons. But we choose the average ftness rather than best ftness as the stoppng crtera. 5 Smuaton study hs secton present some evauaton resuts of proposed dentfer for moduaton set that s defned n secton. For smpfyng the ndcaton, we substtute the moduatons ASK4, ASK8, PSK, PSK4, PSK8, Star-QAM8, V9, QAM6, QAM64,
Proceedngs of the 6th WSEAS Internatona Conference on Apped Informatcs and Communcatons, Eounda, Greece, August 8-0, 006 (pp388-393) QAM8 wth P, P, P 3, P 4, P 5, P 6, P 7, P 8, P 9, P 0 respectvey. A sgna are dgtay moduated n MALAB smuator. he smuated sgnas were aso band-mted and Gaussan nose was added accordng to SNRs, 3, 0, 3, 6, 9,, and 8 db. Each moduaton type has 00 reazatons of 048 sampes. Fgure shows the herarcha SVM-based cassfer. abe shows the chosen features for each SVM that dscrmnate sgna types wth them. At the frst, we evauate the performance of system wthout GA. Based on some smuatons, the vaue σ = s seected for a SVMs. he effect of nose on cassfer performance s aso studed wth dfferent SNRs. Random ony 0% of data s used for tranng. Overa performance (OP) s shown n abe. As we see, the performance s generay good even wth ow SNR vaues. Now, we appy the GA or mode seecton of SVMs. We have used the GA method for each of SVM separatey. One of the advantages of herarchca mutcass-svm based n comparson others mut-cass SVM-based methods (.e. one-agansta and one-aganst-one) s that optmzaton of each SVM coud be done separatey. In the other methods even though each SVM s tuned very we for the bnary probem, there s no guarantee that they work we together for the entre probem and the parameters of the kernes affect the structure of the feature space and the cassfcaton accuracy. abe 3, shows the overa performance of proposed dentfer. As we see, mode seecton exacty mproves the performances of system n a SNRs; especay n ower SNRs. SVM P 3 SVM SVM4 SVM6 SVM3 SVM5 P 6 P P P 4 P 5 abe 3: OP wth appyng of GA for mode seecton at dfferent SNRs. SNR (db) OP (%) -3 89.8 0 93.7 3 94.93 6 97.35 9 98.43 8 98.76 SVM7 P 0 P 7 SVM8 SVM9 Fgure. Herarchca SVM-Based Cassfer abe : Chosen features for each SVM P 8 P 9 SVM s Number Chosen features SVM C 83 SVM M 4 SVM 3 M 4 SVM 4 C 80 SVM 5 C 80, M 6 SVM 6 C 63 SVM 7 C 8 SVM 8 C 63 SVM 9 C 80, M 84 abe : OP wthout SVMs mode seecton at dfferent SNRs. SNR (db) OP (%) -3 84.3 0 9.67 3 9. 6 94. 9 96.88 8 98.05 For comparson our method wth other methods as sad there are a few researches on moduaton dentfcaton that consder the number of moduatons about ten or a tte and most of methods that have been proposed have mtaton to consder addtona moduaton. hs s many because of ther features and/or cassfers. In [0], the author reported a generazaton rate of 90% and 93% of accuracy of data sets wth SNR of 5 5 db for consdered moduaton set. However, the performances for ower SNRs are reported to be ess than 80% for a fuy connected network, and about 85% for a herarchca network. In [], the authors deveoped and compared one decson tree and one neura network cassfer. Haf of the smuated sgnas used for tranng had a SNR vaue of 5 db and the other haf 0 db. It used nstantaneous features n addton spectra features. he neura network cassfer conssted of three MLPs. he resuts showed 88% success rate at 5 db SNR for consdered moduatons set. In [], the researches show the average dentfcaton rate s 83%, and t reaches over 90% for SNR vaue of over 0 db. However, f SNR s ess than 0 db, the performance drops to ess than 70%. he dentfer proposed n ths work shows a steady performance wth dfferent SNR vaues and has a hgh performance at very ow SNRs. Usng an opt-
Proceedngs of the 6th WSEAS Internatona Conference on Apped Informatcs and Communcatons, Eounda, Greece, August 8-0, 006 (pp388-393) mzaton method, the performance of dentfer ncrease. he structure of proposed method s very smpe. he proposed method has a hgh generazaton abty and accuracy. 6 Concusons Automatc sgnas type cassfcaton (ASC) has seen ncreasng demand n both mtary and cvan. Most of prevous technques can dentfy ony a few knds of sgna types and usuay need hgh SNR to acheve a mnmum acceptabe performance and don t ncude hgher order and new types of sgnas. he work presented here proposes nove hgh performance method for dentfcaton of dgta sgna types. In ths method we have used a herarchca mutcass cassfer based on support vector machne (SVM). he nputs of ths cassfer are hgher order moments and cumuants (up to eght). he SVM cassfer, use the feature vector and maps the nput vectors non-nearty nto hgh dmensona feature space and constructs the optmum separatng hyper-pane n the space to reaze sgna recognton. hs method s robust and avods over-fttng and oca mnmum. Optmzaton usng GA, mproves the performance of system especay n ower SNRs. References: [] C. Le Martret, and D. Botea, A genera max mum kehood cassfer for moduaton cassfcaton, Proc. ICASSP, Vo. 4, 998, pp. 65-68. [] J. A. Ss, Maxmum-kehood moduaton cassfcaton for PSK/QAM, Proc. MILCOM, 999, pp. 57-6. [3] P. Panagotou, and A. Poydoros, Lkehood rato tests for moduaton cassfcatons, Proc. MILCOM, 000, pp. 670-674. [4] K.C. Ho, W. Prokopw and Y.. Chan, Moduaton dentfcaton of dgta sgnas by waveet transform, Proc. IEE,Radar, Sonar Navg., Vo.47, No.4, 000,pp. 69-76. [5] B. G. Mobas, Dgta moduaton cassfcaton usng consteaton shape, Sgna Processng, Vo. 80, 000, pp. 5 77. [6] J. Lopatka, and P. Macre, Automatc moduaton cassfcaton usng statstca moments and a fuzzy cassfer, Proc. ICSP, 000, pp.- 7. [7] A. Swam, and B. M. Sader, Herarchca dgta moduaton cassfcaton usng cumuants, IEEE rans. Comm., Vo. 48, No. 3, 000, pp. 46 49. [8] S. A. Seyedn, and, A. Ebrahmzadeh, Automatc sgna type dentfcaton by neurawaveet based method, Proc. SICED, 005, pp. 350-355. [9] A. Ebrahmzadeh, S. A. Seyedn, A new method for dentfyng of sgna type, Proc. CIS, 005, 56-6. [0] C. L. P. Seher, Automatc moduaton recognton wth a herarchca neura network, Proc. MILCOM, 993, pp. 5. [] A.K. Nand, E.E. Azzouz, Agorthms for automatc moduaton recognton of communcaton sgnas, IEEE rans. Commun., Vo. 46, No. 4, 998, pp. 43 436. [] L. Mngquan, X. Xanc, L. Lemng, Cycc spectra features based moduaton recognton, Proc. Comm. ech., Vo., 998, pp. 79 795. [3] A. Ebrahmzadeh, and S. A. Seyedn, Identfcaton of sgna type usng svm n fadng envronment, Proc. ICEE, 006. [4]A. Ebrahmzadeh, and S. A. Seyedn, An Integent echnque for Identfcaton of Dgta Moduatons, accepted to pubsh n WSEAS rans Comp., 006. [5] C. Cortes, and V. Vapnc, Support Vector Network, Machne Learnng, Vo. 0, 995, pp.- 5. [6] C. Burges A utora on Support Vector Machnes for Pattern Recognton, Data Mnng and Knowedge Dscovery, Vo., 998, pp -67. [7] B. SchOkopf, C. Burges, and V. Vapnk, Extractng support data for a gven task, ICKDDM,995, pp. 5-57. [8] O. Chapee, V. Vapnk, O. Bousquet, and S. Mukheree, Choosng mutpe parameters for support vector machnes, Machne Learnng, 00, Vo. 46, No., pp. 3-59. [9] Y. X. Su, B. Y. Duan, C. H. Zheng, Genetc desgn of knematcay optma fne tunng Stewart patform for arge spherca rado teescope, Mechatroncs, 00, Vo., No.7, pp. 8-835.