An Analysis on Link Structure Evolution Pattern of Web Communities

Transcription

1 DEWS C HITS An Analysis on Link Structure Evolution Pattern of Web Communities Noriko IMAFUJI and Masaru KITSUREGAWA Institute of Industrial Science, University of Tokyo Komaba 4-6-1, Meguro-ku, Tokyo, Japan Abstract In this paper, we analyze the growing process of link structure of web communities, which is an approach for understanding evolution of the web. A web community is a set of web pages created by individuals or associations with a common interest on a topic. These pages are co-cited by multiple pages, and form densely connected link structure. We examine the transition of link structure within a certain period of time using four sets of web communities created from Japanese web archives crawled in four periods between 1999 and Especially, we focus on the web communities which did not exist in the previous year and classify the evolution of link structures into some patterns. We analyze the semantics of each evolution pattern on the web. Key words web community, web graph, link structure, HITS 1.

2 Fig. 1 The bow-tue structure of the web [6], [7] Borodin 92% 3 [6] 3 ( 1 Albert 2 19 [7] [8][10], [17] [8], [9] [10] [17] 4 HITS HITS [2], [3], [11][13] [13] 2 2 HITS [1][4] 1 [11], [12] (maximum-flow algorithm) [14][16] Max-Flow 2 (a) 2 HITS (b) Max-Flow 1() ()

3 HITS 2 ; (a)hits (b)max-flow Fig. 2 Communities and link structures by HITS family method(a) and Max-Flow method(b) HITS [5][5] HITS 3 (ADG) 4 ADG (SDG) 5 SDG (ADG) Companion- [5] HITS u v u Companion- v ADG uv uv (SDG) SDG 3. 2 t 1,..., t n 3 t i C(t i)={c 1(t i),c 2(t i),..., c m(t i)} C(t i) t i c + (t i) 1 : c + (t i) > = < = n < = 4 2 : c + (t i) 50% URL 3 : c + (t i) t i 1 URL 80% C(t i 1) c j(t i 1) C(t i 1) c j(t i 1) < = 2 12 C(t i) 1 URL 2 URL A = {a 1,a 2,..., a l } c H = {h 1,h 2,..., h m} A 2 E A H 2 G c(v,e) c c (x, Y ) x Y H o = {x x H, (x, A) > (x, A) } H i = {x x H, (x, A) < = (x, A) } H o H i = H, H o H i = 3 G c(v,e) V = H A = {a 1,..., a 6,h 1,..., h 5} H o A A

4 H i A A H i A 3 G c(v,e) Fig. 3 Link structure of web communities, G c(v,e) H oh i H oh i H o H i PC A = {a 1,a 2,..., a m} PC A A H o H i PC A H i H o H i Table 1 Link structure evolution pattern t i 1 t i 1 H o > H i H o > H i 2 H o > H i H o < = H i 3 H o < = H i H o > H i 4 H o < = H i H o < = H i 1 4 Fig. 4 Examples of link evolution patterns ( 5 )

5 3 Table 3 Details of web community sets (a) (b) Fig. 5 Intuive graph of evolution pattern ( jp ) 4 URL URL URL URL 2 2 Table 2 Details of web archives URL M 34M 120M M 32M 112M M 76M 331M M 84M 375M ( ) URL % Fig. 6 Rate of the member pages existed in the previous year 7 t i (a)(b) ( 1 2 3) t i 1 < = i < = 4 t 1,t 2,t 3,t Fig. 7 Rate of members within depth 2. 42

6 H o(t i 1) t i 1 H i(t i 1) t i 1 H o(t i) t i H i(t i) ; t i H o > H i H o < = H i H o > H i t i %, t i 85.32% (t i 1 ) 7 85% 4 (a) H o(t i 1 ), (b) H i (t i 1 ) Table 4 Ave. of (a) H o(t i 1 ), (b) H i (t i 1 ) with respect to each link evolution pattern (a) (b) (a) (b) (a) (b) (a) (b) t 2 =>t t 3 =>t t 4 =>t Average H o(t i) H i(t i) 5 (a) H o(t i ), (b) H i (t i ) Table 5 Ave. of (a) H o(t i ), (b) H i (t i ) with respect to each link evolution pattern (a) (b) (a) (b) (a) (b) (a) (b) t 2 =>t t 3 =>t t 4 =>t Average H o H i Fig. 8 Comparison between H o and H i % 3 18% 4 11% 2 4% 62% 38% 9 Fig. 9 Pacentages of link evolution patterns. 4 H o(t i 1) H i(t i 1) 4, Fig. 10 Intuitive graph of link evolution patterns % 6 (1)

7 (2) (3) (5) (4 )2001 H o Table 6 An example of evolution pattern % colonolog (1) (2,5) PostgreSQL (3)HTTP (4) H i H i H i jp.com.com 7 4 Table 7 An example of evolution pattern Wide Web/Servers/Log Analysis Tools/ 3 php.html Management/Log analysis/

8 [1] J.M.Kleinberg: Authoritative Sources in a Hyperlinked Environment, Proc. 9th ACM-SIAM Symposium on Discrete Algorithms, pp , [2] D. Gibson, J. M. Kleinberg, and P. Raghavan. Inferring web communities from link topology. In UK Conference on Hypertext, pages , [3] S. Chakrabarti, B. Dom, D. Gibson, J. Kleinberg, P. Raghavan, and S. Rajagopalan. Automatic resource list compilation by analyzing hyperlink structure and associated text. In Proceedings of the 7th International World Wide Web Conference, [4] S. R. Kumar, P. Raghavan, S. Rajagopalan, and A. Tomkins. Extracting large-scale knowledge bases from the web. In The VLDB Journal, pages , [5] M. Toyoda and M. Kitsuregawa. Creating a web community chart for navigating related communities. 12th ACM Hypertext, pages , [6] A. Broder, R. Kumar, F. Maghoul, P. Raghavan, A. Rajagopalan, R. Stata, A. Tomkins,and J. Wiener, Graph strucutre in the web, In Proc. of 9th WWW Conference, 2000 [7] R.Albert, H. Jeong, and A. L. Barabasi, Diameter of the world wide web Nature, 401:130, 1999 [8] B.E.Brewington and G.Cybenko, How dynamic is the web?, In Proc. of 9th WWW Conference, 2000 [9] J. Cho and H. Garcia-Molina. The evolution of the web and implications for an incre-mental crawler, In Proc. of 26th VLDB, 2000 [10] K.Bharat, B. W. Chang, M, Henzinger and M.Ruhl. Who Links to Whom: Mining Linkage between Web Sites, In Proc. of IEEE ICDM, [11] G. Flake, S. Lawrence, and C. L. Giles. Efficient identification of web communities. In 6th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages , [12] G. W. Flake, S. Lawrence, C. L. Giles, and F. Coetzee. Selforganization of the web and identification of communities. IEEE Computer, 35(3):66 71, [13] R.Kumar, P.Raghavan, S.Rajagopalan, and A.Tomkins, Trawling the web for emerging cyber-communities, In Proc. 8th WWW Conference, [14] R.K.Ahuja, T.L.Magnanti, and J.B.Orlin, Network Flows : Theory, Algorithms, and Applications, Prentice Hall, Englewood Cliffs, NJ, [15] A.V.Goldberg and R.E.Tarjan, A new approach to the maximal flow problem, In Proc. 18th Ann. ACM Symposium on Theory of Computing, [16] L.R.Ford Jr. and D.R.Fulkerson, Maximal flow through a network, Canadian J.Math.,8: , [17] M.Toyoda and M.Kitsuregawa.:Extracting evolution of web communities from a series of web archives. In Proc. of 14th Conference on Hypertext and Hypermedia(Hypertext 03), pp.28-37, 2003.