Inho Kang. Naver Corporation DEIM Forum 2018 D5-1 ( ) URL PC NAVER 2

HTML
DOWNLOAD

Μέγεθος: px

Εμφάνιση ξεκινά από τη σελίδα:

Download "Inho Kang. Naver Corporation DEIM Forum 2018 D5-1 ( ) URL PC NAVER 2"

Άνεμονη Δημαράς
6 χρόνια πριν
Προβολές:

1 DEIM Forum 2018 D5-1 Inho Kang Naver Corporation ( ) URL PC NAVER , ,,,, URL ( ) 1. Agrawal URL 2015 Google more Google searches were completed on mobile devices than desktop computers. 1 NAVER 2 NAVER URL ( ) NAVER [2], [12], [16] [3], [15] 3 Agrawal [1] URL ( ) URL URL URL URL NAVER ( [5], [14]) NAVER ( ) PC [11] NAVER URL ( ) ( ) 3 3 ACM CIKM 2017 short paper [9]

1 URL 1 URL A, URL B, URL C, URL D URL B Agrawal URL B URL 1 Agrawal 2. 2. 1 URL 2. 2 2. 3 ( ) 2.

2 1 URL 1 URL A, URL B, URL C, URL D URL B Agrawal URL B URL 1 Agrawal URL ( ) 2. 1 URL Joachims [7] URL Agrawal [1] URL Agrawal 1 URL URL ( ) 2 (URL) 3 URL URL Agrawal URL URL URL [7] URL ( URL ) URL URL ( ) URL B URL A, URL C 1 URL D 1 Joachims 2 URL 1 URL P (2, 1) URL URL URL B > URL A, URL B > URL C, URL B > URL D. URL B > URL A P (2, 1) URL B > URL D P (2, 4)( ) 1 URL B > URL A P (2, 1) URL B > URL D P (2, 4) Agrawal (4. 1 ) Kadotami [8] ( ) Agrawal Agrawal Kadotami 2 Agrawal -order [1] 1 URL B > URL A URL B URL A

2 -order 4. 2 2. 2 [4], [13], [15] Williams [15] ( ) Lagun [10] Guo Song [6] ( ) 2.

1 1: 3 3 200 B A 200 C c( C) cardscore(c) 4 S cardscore(c) cardscore(c) = s S score(c, s),

3 2 -order [4], [13], [15] Williams [15] ( ) Lagun [10] Guo Song [6] ( ) 2. 3 Lagun [10] URL ( ) ( ) 2 ( ) : B A 200 C c( C) cardscore(c) 4 S cardscore(c) cardscore(c) = s S score(c, s), (1) score(c, s) = time(c, s) dominance(c, s) completeness(c, s), time(c, s) = c s, dominance(c, s) = c s s completeness(c, s) = c s c,.

4 cardscore(c) : order 6 3 A B C 5. 5 ( ) ( ) c C Agrawal Kadotami 5 ( ) cardscore(c) NAVER NAVER NAVER USER ID, SERP ID, QUERY, UNIXTIME, INTERACTION TYPE, VISIBLE ITEMS, CLICKED CARD VISIBLE ITEMS NAVER > > = > = 0.2 < %

5 7, , , , NAVER Lancers ,476 8 (34-1 Jan-34 ) ( ) N1 N2 2 NN 1 L-L-R N1 N2 NN 3 2 NN 2 NN 2 Fleiss κ Cohen s κ [17] Fleiss κ 3 Cohen s κ 2 95% ,023

6 1 N1-N2-NN L-L-L 993 (40.17%) L 1,742 (70.47%) L-L-R 506 (20.47%) L-R-L 105 (4.25%) R-L-L 138 (5.58%) R-R-R 347 (14.04%) R 730 (29.53%) R-R-L 177 (7.16%) R-L-R 92 (3.72%) L-R-R 114 (4.61%) 2,472 (100%) total 2,472 (100%) 2 3 Fleiss κ 2 Cohen s κ κ 95% CI [0.539, 0.584] N1 N [0.551, 0.588] N1 NN [0.178, 0.219] N2 NN [0.224, 0.265] 5. 3 = n N = n N N: N (< = N): -order 2 N N n(< = N ): 5. 4 C (Click) A (Abandonment) score cardscore random C+A(score) [] cardscore C+A(random) C A(score) cardscore A(random) C(score)+A(score) cardscore Tukey HSD 3 ( 3 ) n N N C+A(score) C+A(random) C A(score) A(random) C(score)+A(score) ( 2 ) n N N C+A(score) C+A(random) C A(score) A(random) C(score)+A(score) C A(score) cardscore(c) 3 4 C+A(score) C N 3 5 (p = )

7 5 Tukey HSD ( 3 ) α = 0.05 p p C+A(score) A(random) [ , ] A(score) A(random) [ , ] C(score)+A(score) A(random) [ , ] C A(random) [ , ] C+A(random) A(random) [ , ] C+A(score) C+A(random) [ , ] A(score) C+A(random) [ , ] C(score)+A(score) C+A(random) [ , ] C+A(score) C [ , ] C C+A(random) [ , ] A(score) C [ , ] C+A(score) C(score)+A(score) [ , ] C(score)+A(score) C [ , ] C+A(score) A(score) [ , ] A(score) C(score)+A(score) [ , ] C+A(score) A(score) C+A(score) 3 4 C+A(score) C(score)+A(score) cardscore(c) 5 (p = ) 3 4 C+A(score) C+A(random) 5 p A(score) A(random) 5 p cardscore(c) 3 4 A(random) cardscore cardscore time, dominance, completeness (4. 1 ) 6 cardscore 1 2 A(score) cardscore A(score) 3 t d c time dominance completeness td time dominance Tukey HSD 7 c t completeness time 6 completeness time 7 ( c t ) (p = ) time completeness (tc) (tdc) time (t) ( p = , p = ) completeness 6 cardscore A(score) ( 3 ) time dominance completeness t d c td dc tc tdc

8 7 A(score) cardscore Tukey HSD ( 3 ) α = 0.05 p p c t [ , ] tc t [ , ] tdc t [ , ] dc t [ , ] c d [ , ] c td [ , ] td t [ , ] tc d [ , ] tdc d [ , ] d t [ , ] c dc [ , ] tc td [ , ] tdc td [ , ] dc d [ , ] c tdc [ , ] c tc [ , ] dc td [ , ] tc dc [ , ] tdc dc [ , ] td d [ , ] tc tdc [ , ] time, dominance, completeness cardscore (time) (completeness) [1] Rakesh Agrawal, Alan Halverson, Krishnaram Kenthapadi, Nina Mishra, and Panayiotis Tsaparas. Generating labels from clicks. In Proceedings of ACM WSDM 2009, pp , [2] Jaime Arguello, Fernando Diaz, and Jamie Callan. Learning to aggregate vertical results into web search results. In Proceedings of ACM CIKM 2011, pp , [3] Lydia B. Chilton and Jaime Teevan. Addressing people s information needs directly in a web search result page. In Proceedings of WWW 2011, pp , [4] Aleksandr Chuklin and Pavel Serdyukov. Potential good abandonment prediction. In WWW 2012 Companion, pp , [5] Nick Craswell and Martin Szummer. Random walks on the click graph. In Proceedings of ACM SIGIR 2007, pp , [6] Qi Guo and Yang Song. Large-scale analysis of viewing behavior: Towards measuring satisfaction with mobile proactive systems. In CIKM 2016, [7] Thorsten Joachims, Laura Granka, Bing Pan, Helene Hembrooke, Filip Radlinski, and Geri Gay. Evaluating the accuracy of implicit feedback from clicks and query reformulations in web search. ACM TOIS, Vol. 25, No. 2, [8] Yuta Kadotami, Yasuaki Yoshida, Sumio Fujita, and Tetsuya Sakai. Mobile vertical ranking based on preference graphs. In Proceedings of ACM ICTIR 2017, [9] Mami Kawasaki, Inho Kang, and Tetsuya Sakai. Ranking rich mobile verticals based on clicks and abandonment. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, pp , [10] Dmitry Lagun, Chih-Hung Hsieh, and Dale Webster Vidhya Navalpakkam. Towards better measurement of attention and satisfaction in mobile search. In Proceedings of SIGIR 2014, pp , [11] Jane Li, Scott Huffman, and Akihito Tokuda. Good abandonment in mobile and pc internet search. In Proceedings of ACM SIGIR 2009, pp , [12] Ashok Kumar Ponnuswami, Kumaresh Pattabiraman, Qiang Wu, Ran Gilad-Bachrach, and Tapas Kanungo. On composition of federated web search result page: Using online users to provide pairwise preference for heterogeneous verticals. In Proceedings of ACM WSDM 2011, pp , [13] Yang Song, Xiaolin Shi, Ryen White, and Ahmed Hassan Awadallah. Context-aware web search abandonment prediction. In Proceedings of ACM SIGIR 2014, pp , [14] Kuansan Wang, Toby Walker, and Zijian Zheng. PSkip: Estimating relevance ranking quality from web search clickthrough data. In Proceedings of ACM KDD 2009, pp , [15] Kyle Williams, Julia Kiseleva, Aidan C. Crook, Imed Zitouni, Ahmed Hassan Awadallah, and Madian Khabsa. Detecting good abandonment in mobile search. In Proceedings of WWW 2016, pp , [16] Ke Zhou, Thomas Demeester, Dong Nguyen, Djoerd Hiemstra, and Dolf Trieschnigg. Aligning vertical collection relevance with user intent. In Proceedings of ACM CIKM 2014, pp , [17]. :., 2015.

Σχετικά έγγραφα

DEIM Forum 2013 A2-2 606 8501 E-mail: kato@dl.kuis.kyoto-u.ac.jp 1. 2. 1 4 A B C D A B C D A : B :: C : D : :: : : :: : A B C D A= B= C= D= D 3 Turney [20] A B C D A B C D Bollegala [5] Web SVM A B C D