A Pairwise Document Analysis Approach for Monolingual Plagiarism Detection

Transcription

1 A Pairwise Document Analysis Approach for Monolingual Plagiarism Detection

2 Introuction Plagiarism: Unauthorize use of Text, coe, iea, Plagiarism etection research area has receive increasing attention The rapi growth of ocuments in ifferent languages Increase accessibility of electronic ocuments 29/1/2017 2

3 Prototypical Plagiarism Monolingual: Cross-Language: copy or paraphrase inclues translation 29/1/2017 3

4 Problem efinition has two steps Caniate ocument retrieval D: set of source ocuments : suspicious ocument with fragments f Pairwise ocument similarity : source ocument with fragments f : suspicious ocument with fragments f } ), (,, { ), ( f f f f Sim D D ocuments Caniate } ), (,,, { ), ( f f f f f f Sim pairs Copie 29/1/2017 4

5 Detaile analysis in a pair of ocuments Possible errors in etecting plagiarism: Text that is not plagiarize might be erroneously reporte Part or whole of plagiarize source or target text might be unreporte Parts of one plagiarism case might be reporte as separate cases 29/1/2017 5

6 Evaluation Metrics S: set of true plagiarism cases, R: set of etections reporte Precision( R, S) 1 R rr ss ( s r r) Recall( R, S) 1 S ss rr ( s s r) Fraction of reporte etections (at character level) that are truly plagiarize Fraction of plagiarism cases (at character level) that are etecte Granularity(R, S) = 1 S R å sîs R R s Average number of reporte etections per etecte plagiarism case Plaget( R, S) Combine metric log 2 F1 ( R, S) (1 Granularity( R, S)) 29/1/2017 6

7 Two phase algorithm for ientifying plagiarize text fragments Caniate sentence selection: Fins many possibly plagiarize fragments Focusing on recall Result filtering: Fins alignments between the ientifie passages Focusing on precision 29/1/2017 7

8 Step 1: Caniate Sentence Selection 29/1/2017 8

9 Token Extraction Source () Obtain fragments f f Obtain fragments Suspicious ( ) Seeing: Token extraction, Each fragment is create from a sequence of k Using all wors or keywors consecutive sentences using a sliing winow Representative wors K 1 K n K 1 K n 29/1/2017 9

10 Create vector Create vector Token Extraction Check existence of items in fragments Check existence of items in fragments Source () Obtain fragments f f Obtain fragments Suspicious ( ) Match merging: Two etecte fragments are Ientify Use Cosine presence Similarity of representative terms merge to report a single plagiarism case if the number of characters between those fragments in the source an suspicious ocuments are both below a proximity threshol Representative wors K 1 K n K 1 K n sim( f, K 1 ) sim( f, K 1 ) Similarity computation 29/1/2017 sim( sim( 10 f, K n ) f, K n )

11 Create vector Create vector Token Extraction Check existence of items in fragments Check existence of items in fragments Source () Obtain fragments f f Obtain fragments Suspicious ( ) Cross-lingual plagiarism etection Representative wors K 1 K n Translate K 1 K n sim( f, K 1 ) sim( f, K 1 ) Similarity computation 29/1/2017 sim( sim( 11 f, K n ) f, K n )

12 Step 2: Result Filtering 29/1/

13 Aligning segments within fragment pairs Fragment pair from the first step retrieve Fragments split into smaller segments Segments aligne using a ynamic programming algorithm allowing 1:0, 0:1, 1:1, 2:1, 1:2, 3:1 an 1:3 alignments exclue sentences at start or en of fragment with >50% content in 1:0 or 0:1 alignments f 29/1/ f

14 Alignment etails where S(i, j) represents the score of the optimal alignment from the beginning of the fragment to the i th suspicious segment an the j th source segment To penalize 1-0 an 0-1 alignments an also to make all scores comparable, we keep track of the number of alignments obtaine so far, an the score in each step is normalize by the number of alignments 29/1/

15 Granularity level of alignment Sentence level: Using sentences as the granularity level of alignment n-gram level: A plagiarize fragment may omit pieces from the source, but it is likely that at least some of the smallest units are preserve n is the expecte number of terms in each segment 29/1/

16 Results Result of etaile analysis sub-task using PersinaPlaget2016 training corpus t = Similarity threshol, n=number of sentences Precision Recall Granularity Plaget (t = 02, n = 5) (t = 03, n = 5) (t = 04, n = 5) (t = 03, n = 3) (t = 04, n = 3) Result of etaile analysis sub-task using PersinaPlaget2016 test corpus Precision Recall Granularity Plaget Runtime (t = 03, n = 5) :24:08 29/1/

17 Results Evaluation of the secon phase, result filtering step: t = 03, n = 3 Precision Recall Granularity Plaget Without result filtering After result filtering /1/

18 Results Evaluation of the seeing phase, using keywors: Precision Recall Granularity Plaget (t = 03, n = 3) (t = 04, n = 3) (t = 05, n = 3) (t = 06, n = 3) (t = 07, n = 3) /1/

19 Cross-lingual etaile analysis for plagiarism etection Ehsan, N, Tompa, FW, Shakery, A: Using a ictionary an n-gram alignment to improve fine-graine cross-language plagiarism etection In: Proceeings of the 2016 ACM Symposium on Document Engineering pp ACM (2016) Precision Recall Granularity Plaget Using PAN2012 English-German ataset /1/

20 Summary The propose metho is a two phase approach for ientifying plagiarize fragments The first phase tries to fin possibly plagiarize fragments The secon phase tries to improve the precision metric The framework is applicable in any language The approach coul be aapte for cross language omain 29/1/

21 Thanks for your attention 29/1/