Analyzing Similarity of HTML Structures and Affiliate in Automatic Collection of Splogs

HTML
DOWNLOAD

Μέγεθος: px

Εμφάνιση ξεκινά από τη σελίδα:

Download "Analyzing Similarity of HTML Structures and Affiliate in Automatic Collection of Splogs"

Ἀμών Κορωναίος
9 χρόνια πριν
Προβολές:

1 DEIM Forum 2011 F2-2 HTML ( ) HTML ID HTML ID SVM HTML Analyzing Similarity of HTML Structures and Affiliate in Automatic Collection of Splogs Akihito MORIJIRI, Taichi KATAYAMA,SoichiISHII,TakehitoUTSURO, Yasuhide KAWADA, and Tomohiro FUKUHARA College of Engineering Systems, School of Science and Engineering, University of Tsukuba Graduate School of Systems and Information Engineering,University of Tsukuba Graduate School of Science and Technology for Future Life, Tokyo Denki University Navix Co., Ltd. Center for Service Research, National Institute of Advanced Industrial Science and Technology Abstract Spam blogs or splogs are blogs hosting spam posts, created using machine generated or hijacked content for the sole purpose of hosting advertisements or raising the number of in-links of target sites. It has been shown that splogs can be detected based on similarity of HTML structures and affiliate IDs. The similarity of HTML structures of splogs is effective in splog detection, and the identity of affiliate IDs extracted from splogs can identify spammers much more directly than similarity of HTML structures, although it is not easy to achieve high coverage in extracting affiliate IDs. The coverage of the intersection of the two clues, similarity of HTML structures and affiliate IDs, is relatively low, and it is necessary to apply them in a complementary strategy. This paper studies how to detect splogs which cannot be detected based on either similarity of HTML structures nor affiliate IDs. We apply SVMs to this task and show that splogs of above type can be detected with high precision. Key words spam blog detection, HTML structure, affiliate, machine learning 1.

and Tomohiro FUKUHARA College of Engineering Systems, School of Science and Engineering, University of Tsukuba Graduate School of Systems and Information Engineering,University of Tsukuba Graduate

2 Technorati BlogPulse [2] kizasi.jp Globe of Blogs Best Blogs in Asia Directory ( ) [3], [9], [10], [12] [10] 88% 75% [11] [9], [10], [12] [12] TREC Blog06 / [9], [10] BlogPulse [4], [8], [11], [13] HTML HTML ID [7] HTML 1 13 [5] ID ID [6] 2 HTML ID ( 1 4) Support Vector Machines (SVMs) [16] 2. HTML 2. 1 HTML DOM [18] HTML DOM [7] 2 HTML s HTML HTML P DIV P DIV P DIV [18] BODY P DIV BODY HTML [18] SCRIPT STYLE HTML HTML s DOM dm(s) 2. 2 DOM HTML s t DOM dm(s) dm(t) DP DP 1 2 DP ( ) edit distance (dm(s),dm(t)) DOM dm(s) dm(s) s t DOM Rdiff(s, t) Rdiff(s, t) = edit distance (dm(s),dm(t)) dm(s) + d m(t) HTML s HTML T t T DOM Rdiff(s, t) MinDF(s, T ) MinDF(s, T ) = min Rdiff(s, t T ) t =s 3. ID 3 ID ID ID ID ID [5] ASP( ) ID 10 1 ID 1 Am At D Gl I Lk R St Tr V

[5] ID ID 1 23 2 [6] 2 HTML ID ( 1 4) Support Vector Machines (SVMs) [16] 2. HTML 2.

3 1 KANSHIN [1] RSS Atom S F 2 11 S 48,183 14,352 ( 30%) F 60,977 6,231 ( 10%) ID ASP ID ID ID ID ID ID [5] ID ID [5] ID ASP10 ID 20,583 (S F ) ID ID H ID SP af (H, x) x 3 S 72 1,101 F ID ID 129 ID ID 2,472 ID ID 121 ID 2,054 ( ID 93.8% 83.3%) ID ID ID ID ID

) 10 1 10 ID ID H ID SP af (H, x) x 3 S 72 1,101 F 56 953 ID ID 129

4 1 2 HTML DOM DOM ID x H ID x SP af (H,x) ID ID 1 4. SVM 4. 1 / URL / HTML URL URL i) HTML URL ii) HTML 2 URL URL u URL log u u u URL 4. 2 [14], [17] / 3 / w w φ 2. ID ID 3 ( IPAdic (

5 φ 2 (, w)= w w freq(, w freq(, w )=a )=b freq(, freq(, w) =c w) =d (ad bc) 2 (a + b)(a + c)(b + d)(c + d) ( log φ 2 (, w) w w 4. 3 URL / URL URL ( ) w s AncfB(w, s) AncfW (w, s) s w AncfB(w, s) = URL s w AncfW (w, s) = URL AncfB(w, s) 2 s URL w URL t t URL ( ) log AncfB(w, s) AncfB(w, t) w s AncfW (w, s) 2 s URL w URL t t URL ( ) log AncfW (w, s) AncfW (w, t) w s 5. SVM TinySVM ( ) SVM [15] LBD s LBD ab S 48,183 F 60, ID 1 S 1,101 F 953 ID ID ID ID ID S 72 ID F 56 ID ID ID 500 ( ) SVM ID ID S 13 ID F

AncfB(w, s) AncfB(w, t) w s AncfW (w, s) 2 s URL w URL t t URL ( ) log AncfW (w, s) AncfW (w, t) w s 5. SVM TinySVM (http://chasen.

6 (a) 4 (S ) (b) (a) 5 (F ) (b) 12 ID ID ID 500 ( ) SVM S 48,183 F 60, ID 10 ID HTML (MinDF > 0.15) S 47,029 F 59, S 4 ID 283 F 268 / 6. 3 LBD s LBD s LBD s S 4(a) F 5(a) LBD ab LBD ab LBD ab S 4(b) F 5(b) 4 5 ID ID ID ID

15) S 47,029 F 59,982 1 4 S 4 ID 283 F 268 / 6.

7 ID ID ID 4(a) 5(a) S 35% F 60% 90% 4(b) 5(b) S 17% F 15% 90% HTML ID HTML [7] ID [5] 7. HTML ID HTML ID SVM [1],,,.. 21, [2] N. Glance, M. Hurst, and T. Tomokiyo. Blogpulse: Automated trend discovery for Weblogs. In Proc. Workshop on the Weblogging Ecosystem: Aggregation, Analysis and Dynamics, [3] Z. Gyöngyi and H. Garcia-Molina. Web spam taxonomy. In Proc. 1st AIRWeb, pp , [4].. Letters, Vol. 6, No. 4, pp , [5],,,. ID. Web (WebDB2010)., [6],,,,,. HTML. Web (WebDB2010)., November [7],,,,. HTML. 2 DEIM, [8] P. Kolari, T. Finin, and A. Joshi. SVMs for the Blogosphere: Blog identification and Splog detection. In Proc AAAI Spring Symp. Computational Approaches to Analyzing Weblogs, pp , [9] P. Kolari, T. Finin, and A. Joshi. Spam in blogs and social media. In Tutorial at ICWSM, [10] P. Kolari, A. Joshi, and T. Finin. Characterizing the splogosphere. In Proc. 3rd Workshop on the Weblogging Ecosystem: Aggregation, Analysis and Dynamics, [11] Y.-R. Lin, H. Sundaram, Y. Chi, J. Tatemura, and B. L. Tseng. Splog detection using self-similarity analysis on blog temporal dynamics. In Proc. 3rd AIRWeb, pp. 1 8, [12] C. Macdonald and I. Ounis. The TREC Blogs06 collection : Creating and analysing a blog test collection. Technical Report TR , University of Glasgow, Department of Computing Science, [13] G. Mishne, D. Carmel, and R. Lempel. Blocking blog spam with language model disagreement. In Proc. 1st AIRWeb, [14] Y. Sato, T. Utsuro, T. Fukuhara, Y. Kawada, Y. Murakami, H. Nakagawa, and N. Kando. Analyzing features of Japanese splogs and characteristics of keywords. In Proc. 4th AIRWeb, pp , [15] S. Tong and D. Koller. Support vector machine active learning with applications to text classification. In Proc. 17th ICML, pp , [16] V. N. Vapnik. Statistical Learning Theory. Wiley- Interscience, [17] Y.M. Wang, M. Ma, Y. Niu, and H. Chen. Spam doublefunnel: Connecting web spammers with advertisers,. In Proc. 16th WWW, pp , [18],.., Vol. 8, No. 1, pp , 2009.

In Proc. 1st AIRWeb, pp. 39 47, 2005. [4].. Letters, Vol. 6, No. 4, pp. 37 40, 2008. [5],,,. ID. Web (WebDB2010)., 2010. [6],,,,,. HTML. Web (WebDB2010)., November 2010. [7],,,,. HTML. 2 DEIM, 2010.

Σχετικά έγγραφα

HTML. Evaluating Effects of Similarities of HTML Structures in Splog Detection

HTML. Evaluating Effects of Similarities of HTML Structures in Splog Detection HTML 1 2 1 3 4 ( ) HTML HTML DOM SVM Evaluating Effects of Similarities of HTML Structures in Splog Detection Taichi Katayama, 1 Takayuki Yoshinaka, 2 Takehito Utsuro, 1 Yasuhide Kawada 3 and Tomohiro