HTML Utilizing Similarities of HTML Structures in Splog Detection by Machine Learning

Μέγεθος: px

Εμφάνιση ξεκινά από τη σελίδα:

Download "HTML Utilizing Similarities of HTML Structures in Splog Detection by Machine Learning"

Κρειος Δοξαράς
5 χρόνια πριν
Προβολές:

1 2009 Technical Report on Information-Based Induction Sciences 2009 (IBIS2009) HTML Utilizing Similarities of HTML Structures in Splog Detection by Machine Learning Taichi Katayama Takayuki Yoshinaka Takehito Utsuro Yasuhide Kawada Tomohiro Fukuhara Abstract: Spam blogs or splogs are blogs hosting spam posts, created using machine generated or hijacked content for the sole purpose of hosting advertisements or raising the PageRank of target sites. Among those splogs, this paper focuses on detecting a group of splogs which are estimated to be created by an identical spammer. We especially show that similarities of html structures among those splogs created by an identical spammer contribute to improving the performance of splog detection. In measuring similarities of html structures, we extract a list of blocks (minimum unit of content) from the DOM tree of a html file. We show that the html files of splogs estimated to be created by an identical spammer tend to have similar DOM trees and this tendency is quite effective in splog detection. Keywords: spam blog, machine learning, HTML structure, confidence measure, SVM 1, , {katayamataichi2008, nlp.iit.tsukuba.ac.jp Graduate School of Systems and Information Engineering,University of Tsukuba, Tsukuba, , Japan, , cdl.im.dendai.ac.jp Graduate School of Engineering, Tokyo Denki University, Tokyo, , Japan ( ), , Navix Co., Ltd., Nishi-Gotanda, Shinagawa-Ku Tokyo , Japan, , race.u-tokyo.ac.jp Research into Artifacts, Center for Engineering, University of Tokyo Kashiwa, Chiba , Japan Technorati 1 BlogPulse 2 kizasi.jp 3 blogwatcher 4 [1] Globe of Blogs 5 Best Blogs in Asia Directory 6 Blogwise 7 ( ) [2,3,4,5,6] ( ) 4 ( )

1: / 1 (a) / CC SS 198 293 277 768 210 259 2849 3318 408 552 3126 4086 (b) CC SS ID 1 2 3 4

Support Vector Machines [11] (SVM) SVM SVM [12] HTML SVM 8 http://trec.nist.

2 1: / 1 (a) / CC SS (b) CC SS ID [4] 88% 75% [3, 7] [5] TREC 8 Blog06 / [4, 6] BlogPulse [8, 9, 10, 4] HTML Support Vector Machines [11] (SVM) SVM SVM [12] HTML SVM 8 1: HTML DOM DOM 2 / / / [13, 14] / / [13, 14] ID ID 1 CC ID=1 SS ID= 2, 3, 4 [13, 14] 2 HTML

3 3 HTML 3.1 HTML DOM [15] HTML DOM 1 HTML s HTML HTML P DIV P DIV P DIV [15] BODY P DIV BODY HTML [15] SCRIPT STYLE HTML HTML s DOM dm(s) 3.2 DOM HTML s t DOM dm(s) dm(t) DP DP 1 2 DP edit distance (dm(s),dm(t)) s t DOM Rdiff(s, t) Rdiff(s, t) = edit distance (dm(s),dm(t)) dm(s) + dm(t) 1 HTML DOM 3.3 DOM HTML S T HTML s S t T DOM DOM HTML s S HTML T t T Rdiff(s, t) 10 AvMinDF 10 (s, T ) 9 AvMinDF 10 (s, T ) = T Rdiff(s, t T ) 10 t Rdiff(s, t) (ID=1, CC ) (ID=2, 3, 4, SS ) DOM 2 ID ( ID=1) S T S s AvMinDF 10 (s, T ) (ID=1, CC ) ID=1 S CC T S s AvMinDF 10 (s, T ) SC SS 2(a) CC S T S s AvMinDF 10 (s, T ) 2(b) (c) (d) SS ( 2(b) (c) (d) ) 2(a),(c),(d) 2(a) AvMinDF 10 (s, T ) AvMinDF k (s, T ) k =1,...25 k =10

4 (a) (ID = 1 CC ) (b) (ID = 2 SS ) (c) (ID = 3 SS ) (d) (ID = 4 SS ) 2: DOM 2(b) ID=2, AvMinDF 10 (s, T ) ID HTML DOM 4 SVM 4.1 DOM 3 HTML DOM s 3.3 AvMinDF 10 (s, T ) log AvMinDF 10 (s, T ) T 1. T DOM ( ) 2. T DOM ( ) 4.2 [16, 17] / URL / HTML URL URL i) HTML URL ii) HTML 2 URL

5 URL u URL log u u u [18, 13, 14] / 10 / w w φ 2. w w freq(, freq(, w )=a w )=b freq( freq(, w) =c φ 2 (, w) =, w) =d (ad bc) 2 (a + b)(a + c)(b + d)(c + d) log ( ) φ 2 (, w) w w URL / URL URL ( ) w s AncfB(w, s) AncfW (w, s) s w Ancf B(w, s) = URL 10 ( sourceforge.jp/) ipadic s w AncfW (w, s) = URL AncfB(w, s) 2 s URL w URL t t URL log w ( s ) Ancf B(w, s) AncfB(w, t) AncfW (w, s) 2 s URL w URL t t URL log ( ) AncfW (w, s) AncfW (w, t) w s SVM SVM TinySVM (

6 5.2 SVM [12] 11 LBD p LBD n (a) CC SS / ( 3) / ( 4) /CC 408 SS 552 / CC SS / / LBD p LBD p LBD p LBD n LBD n 11 ( [19, 12, 20] ) LBD n DOM ( ) DOM ( ) +DOM ( ) DOM 3 4 (a-1) (a-2) 2 DOM ( ) URL DOM ( ) 3 4 (b-1) (b-2) 2 DOM ( ) URL URL DOM ( ) 3 (a-1) (b-1) (a-2) (b-2) DOM ( ) (a-2) DOM ( ) 4 DOM ( ) DOM ( ) DOM ( ) +DOM ( ) DOM ( ) +DOM ( ) 4/5 +DOM ( ) +DOM ( )

7 (a-1) (CC ) (a-2) (CC ) (b-1) (SS ) (b-2) (SS ) 3: / (a-1) (CC ) (a-2) (CC ) (b-1) (SS ) (b-2) (SS ) 4: /

8 DOM ( ) 2 DOM ( ) DOM ( ) 7 HTML SVM HTML SVM DOM [21] HTML DOM DOM [1] T. Nanno, T. Fujiki, Y. Suzuki, and M. Okumura. Automatically collecting, monitoring, and mining Japanese weblogs. In WWW Alt. 04: Proc. 13th WWW Conf. Alternate Track Papers & Posters, pp , [2] Z. Gyöngyi and H. Garcia-Molina. Web spam taxonomy. In Proc. 1st AIRWeb, pp , [3] Wikipedia, Spam blog. blog. [4] P. Kolari, A. Joshi, and T. Finin. Characterizing the splogosphere. In Proc. 3rd Ann. Workshop on the Weblogging Ecosystem: Aggregation, Analysis and Dynamics, [5] C. Macdonald and I. Ounis. The TREC Blogs06 collection : Creating and analysing a blog test collection. Technical Report TR , University of Glasgow, Department of Computing Science, [6] P. Kolari, T. Finin, and A. Joshi. Spam in blogs and social media. In Tutorial at ICWSM, [7] Y.-R. Lin, H. Sundaram, Y. Chi, J. Tatemura, and B. L. Tseng. Splog detection using self-similarity analysis on blog temporal dynamics. In Proc. 3rd AIRWeb, pp. 1 8, [8].. Letters, Vol. 6, No. 4, pp , [9].. Web (WebDB Forum)2008., [10] P. Kolari, T. Finin, and A. Joshi. SVMs for the Blogosphere: Blog identification and Splog detection. In Proc AAAI Spring Symp. Computational Approaches to Analyzing Weblogs, pp , [11] V. N. Vapnik. Statistical Learning Theory. Wiley- Interscience, [12] S. Tong and D. Koller. Support vector machine active learning with applications to text classification. In Proc. 17th ICML, pp , [13],,,,,,.. DEWS2008, [14] Y. Sato, T. Utsuro, T. Fukuhara, Y. Kawada, Y. Murakami, H. Nakagawa, and N. Kando. Analysing features of Japanese splogs and characteristics of keywords. In Proc. 4th AIRWeb, pp , [15],.., Vol. 8, No. 1, pp , [16],,,,,.. DEIM, [17] T. Katayama, Y. Sato, T. Utsuro, T. Yoshinaka, Y. Kawada, and T. Fukuhara. An empirical study on selective sampling in active learning for splog detection. In Proc. 5th AIRWeb, pp , April [18] Y.M. Wang, M. Ma, Y. Niu, and H. Chen. Spam double-funnel: Connecting web spammers with advertisers,. In Proc. 16th WWW, pp , [19] D. D. Lewis and W. A. Gale. A sequential algorithm for training text classifiers. In Proc. 17th SIGIR, pp. 3 12, [20] G. Schohn and D. Cohn. Less is more: Active learning with support vector machines. In Proc. 17th ICML, pp , [21] J. Suzuki, H. Isozaki, and E. Maeda. Convolution kernels with feature selection for natural language processing. In Proc. 42nd ACL, pp , 2004.

Σχετικά έγγραφα

HTML. Evaluating Effects of Similarities of HTML Structures in Splog Detection

HTML. Evaluating Effects of Similarities of HTML Structures in Splog Detection HTML 1 2 1 3 4 ( ) HTML HTML DOM SVM Evaluating Effects of Similarities of HTML Structures in Splog Detection Taichi Katayama, 1 Takayuki Yoshinaka, 2 Takehito Utsuro, 1 Yasuhide Kawada 3 and Tomohiro