HTML. Evaluating Effects of Similarities of HTML Structures in Splog Detection

HTML
DOWNLOAD

Μέγεθος: px

Εμφάνιση ξεκινά από τη σελίδα:

Download "HTML. Evaluating Effects of Similarities of HTML Structures in Splog Detection"

Ἐφραίμ Κωνσταντίνου
7 χρόνια πριν
Προβολές:

1 HTML ( ) HTML HTML DOM SVM Evaluating Effects of Similarities of HTML Structures in Splog Detection Taichi Katayama, 1 Takayuki Yoshinaka, 2 Takehito Utsuro, 1 Yasuhide Kawada 3 and Tomohiro Fukuhara 4 Spam blogs or splogs are blogs hosting spam posts, created using machine generated or hijacked content for the sole purpose of hosting advertisements or raising the number of inward of target sites. Among those splogs, this paper focuses on detecting a group of splogs which are estimated to be created by an identical spammer. We especially show that similarities of html structures among those splogs created by an identical spammer contribute to improving the performance of splog detection. In measuring similarities of html structures, we extract a list of blocks (minimum unit of content) from the DOM tree of a html file. We show that the html files of splogs estimated to be created by an identical spammer tend to have similar DOM trees and this tendency is quite effective in splog detection. 1. Technorati BlogPulse 5) kizasi.jp blog- Watcher 16) Globe of Blogs Best Blogs in Asia Directory Blogwise ( ) 1),6),11),12),14) 12) 88% 75% 2) 1) 13) 11),12),14) 14) TREC Blog06 / 11) 12) BlogPulse 7) 8) 10) 12) 13) 15) HTML 1 Graduate School of Systems and Information Engineering, University of Tsukuba 2 Graduate School of Engineering, Tokyo Denki University 3 ( ) Navix Co., Ltd. 4 Research into Artifacts, Center for Engineering, University of Tokyo 1 c 2009 Information Processing Society of Japan

1 / (a) / CC SS 198 293 277 768 210 259 2849 3318 408 552 3126 4086 (b) CC SS ID 1 2 3 4 163 26 31 33

/ 2007 9 2008 2 / / 17) / / 17) ID ID 1 CC ID=1 SS ID= 2, 3, 4 17) 2 HTML 1 3. HTML HTML DOM DOM 3.

2 1 / (a) / CC SS (b) CC SS ID HTML HTML HTML DOM DOM DOM DOM Support Vector Machines 21) (SVMs) HTML SVM 2. / / / 17) / / 17) ID ID 1 CC ID=1 SS ID= 2, 3, 4 17) 2 HTML 1 3. HTML HTML DOM DOM 3.1 HTML DOM 23) HTML DOM ) 3) 4) HTML 23) HTML 2 HTML 20) 20) HTML HTML 2 c 2009 Information Processing Society of Japan

3 1 HTML s HTML HTML P DIV P DIV P DIV 23) BODY P DIV BODY HTML 23) SCRIPT STYLE HTML HTML s DOM dm(s) 3.2 DOM HTML s t DOM dm(s) dm(t) DP DP 1 2 DP ( ) edit distance (dm(s),dm(t)) DOM dm(s) dm(s) s t DOM Rdiff(s, t) edit distance (dm(s),dm(t)) Rdiff(s, t) = dm(s) + dm(t) 1 HTML DOM 3.3 DOM HTML S T HTML s S t T DOM DOM HTML s S HTML T t T Rdiff(s, t) 10 AvMinDF 10(s, T ) 1 AvMinDF 10(s, T ) = T Rdiff(s, t T ) 10 t Rdiff(s, t) (ID=1, CC ) (ID=2, 3, 4, SS ) DOM 2 ID ( ID=1) S T S s AvMinDF 10(s, T ) (ID=1, CC ) ID=1 S CC T S s AvMinDF 10(s, T ) SC SS 2(a) CC S T S s AvMinDF 10(s, T ) 2(b) (c) (d) SS ( 2(b) (c) (d) ) 2(a),(c),(d) 2(a) AvMinDF 10(s, T ) (b) ID=2, AvMinDF 10(s, T ) ID HTML 1 AvMinDF k (s, T ) k =1,...25 k =10 3 c 2009 Information Processing Society of Japan

4 (a) (ID = 1 CC ) (b) (ID = 2 SS ) (c) (ID = 3 SS ) (d) (ID = 4 SS ) 2 DOM DOM 4. SVM 4.1 DOM 3 HTML DOM s 3.3 AvMinDF 10(s, T ) log AvMinDF 10(s, T ) 1. T (1) T DOM ( ) (2) T DOM 1 ( ) 4.2 9) / URL / HTML URL URL i) HTML URL ii) HTML 2 URL URL u URL ( ) ( ) u log u u URL ) 22) 4 c 2009 Information Processing Society of Japan

5 / 1 / w w φ 2. w w freq(, w )=a freq(, w )=b freq(, w) =c freq(, w) =d φ 2 (, w)= (ad bc) 2 (a + b)(a + c)(b + d)(c + d) ( ) log φ 2 (, w) w w URL / URL URL ( ) w s AncfB(w, s) AncfW (w, s) s w AncfB(w, s) = URL s w AncfW (w, s) = URL AncfB(w, s) 2 URL w URL 1 ( ipadic s t t URL ( ) log AncfB(w, s) AncfB(w, t) w s AncfW (w, s) 2 URL w URL t t URL ( ) log AncfW (w, s) AncfW (w, t) w s SVM SVM TinySVM ( software/tinysvm/) SVM 19) 2 LBD s LBD ab 2 s 5 c 2009 Information Processing Society of Japan

6 (a-1) (CC ) (a-2) (CC ) CC (b-1) (SS ) (b-2) (SS ) SS 3 / (a-1) (CC ) (a-2) (CC ) (b-1) (SS ) (b-2) (SS ) CC SS 4 / 6 c 2009 Information Processing Society of Japan

7 (a) CC SS / ( 3) / ( 4) / CC 408 SS 552 / CC SS / / LBD p LBD p LBD p LBD n LBD n LBD n DOM ( ) DOM ( ) +DOM ( ) DOM 3 4 (a-1) (a-2) 2 DOM ( ) URL DOM ( ) 3 4 (b-1) (b-2) 2 DOM ( ) URL URL DOM ( ) 3 (a-2) CC DOM ( ) DOM ( ) (a-1) (b-1) (b-2) DOM ( ) DOM ( ) HTML / 3 DOM ( ) DOM ( ) 4 (a-2) SS DOM ( ) (a-1) (b-1) (b-2) DOM ( ) DOM ( ) HTML 4 DOM ( ) DOM ( ) 3 4 DOM ( ) 2 DOM ( ) DOM ( ) 2 7 c 2009 Information Processing Society of Japan

8 7. HTML SVM HTML SVM DOM DOM 24) 18) HTML DOM DOM 1) : Wikipedia, Spam blog. blog. 2) : Wikipedia, Ping (blogging). (blogging). 3) Bing, L., Wang, Y., Zhang, Y. and Wang, H.: Primary Content Extraction with Mountain Model, Proc. 8th IEEE CIT, pp (2008). 4) Debnath, S., Mitra, P., Pal, N. and Giles, C.L.: Automatic Identification of Informative Sections of Web Pages, IEEE Transactions on Knowledge and Data Engineering, Vol.17, No.9, pp (2005). 5) Glance, N., Hurst, M. and Tomokiyo, T.: BlogPulse: Automated Trend Discovery for Weblogs, Proc. WWW 2004 Workshop on the Weblogging Ecosystem: Aggregation, Analysis and Dynamics (2004). 6) Gyöngyi, Z. and Garcia-Molina, H.: Web Spam Taxonomy, Proc. 1st AIRWeb, pp (2005). 7) Letters Vol.6, No.4, pp (2008). 8) Web (WebDB Forum)2008 (2008). 9) Katayama, T., Sato, Y., Utsuro, T., Yoshinaka, T., Kawada, Y. and Fukuhara, T.: An Empirical Study on Selective Sampling in Active Learning for Splog Detection, Proc. 5th AIRWeb, pp (2009). 10) Kolari, P., Finin, T. and Joshi, A.: SVMs for the Blogosphere: Blog Identification and Splog Detection, Proceedings of the 2006 AAAI Spring Symposium on Computational Approaches to Analyzing Weblogs, pp (2006). 11) Kolari, P., Finin, T. and Joshi, A.: Spam in Blogs and Social Media, Tutorial at ICWSM (2007). 12) Kolari, P., Joshi, A. and Finin, T.: Characterizing the Splogosphere, Proc. 3rd Ann. Workshop on the Weblogging Ecosystem: Aggregation, Analysis and Dynamics (2006). 13) Lin, Y.-R., Sundaram, H., Chi, Y., Tatemura, J. and Tseng, B.L.: Splog Detection using Self-similarity Analysis on Blog Temporal Dynamics, Proc. 3rd AIRWeb, pp. 1 8 (2007). 14) Macdonald, C. and Ounis, I.: The TREC Blogs06 Collection : Creating and Analysing a Blog Test Collection, Technical Report TR , University of Glasgow, Department of Computing Science (2006). 15) Mishne, G., Carmel, D. and Lempel, R.: Blocking Blog Spam with Language Model Disagreement, Proc. 1st AIRWeb (2005). 16) Nanno, T., Fujiki, T., Suzuki, Y. and Okumura, M.: Automatically Collecting, Monitoring, and Mining Japanese Weblogs, WWW Alt. 04: Proc. 13th WWW Conf. Alternate Track Papers & Posters, pp (2004). 17) Sato, Y., Utsuro, T., Fukuhara, T., Kawada, Y., Murakami, Y., Nakagawa, H. and Kando, N.: Analysing Features of Japanese Splogs and Characteristics of Keywords, Proc. 4th AIRWeb, pp (2008). 18) Suzuki, J., Isozaki, H. and Maeda, E.: Convolution Kernels with Feature Selection for Natural Language Processing, Proc. 42nd ACL, pp (2004). 19) Tong, S. and Koller, D.: Support Vector Machine Active Learning with Applications to Text Classification, Proc. 17th ICML, pp (2000). 20) Urvoy, T., Lavergne, T. and Filoche, P.: Tracking Web Spam with Hidden Style Similarity, Proc. 2nd AIRWeb, pp (2006). 21) Vapnik, V.N.: Statistical Learning Theory, Wiley-Interscience (1998). 22) Wang, Y., Ma, M., Niu, Y. and Chen, H.: Spam Double-Funnel: Connecting Web Spammers with Advertisers,, Proc. 16th WWW, pp (2007). 23) Vol.8, No.1, pp (2009). 24) HTML (2009). 8 c 2009 Information Processing Society of Japan

Σχετικά έγγραφα

HTML Utilizing Similarities of HTML Structures in Splog Detection by Machine Learning

2009 Technical Report on Information-Based Induction Sciences 2009 (IBIS2009) HTML Utilizing Similarities of HTML Structures in Splog Detection by Machine Learning Taichi Katayama Takayuki Yoshinaka Takehito