1,a) Graham Neubig 1,b) Sakriani Sakti 1,c) 1,d) 1,e) 1. [11], [15] 1 Nara Institute of Science and Technology a) akabe.koichi.zx8@is.naist.jp b) neubig@is.naist.jp c) ssakti@is.naist.jp d) tomoki@is.naist.jp e) s-nakamura@is.naist.jp 1 n-gram 1-gram 2-gram the 61 (BOS) the 42, 47. (EOS) 41 and 43, and 32 of 42 of the 27 : 42 in the 21 1 n-gram n-gram n-gram n-best [16] n-gram n-gram L1 c 2012 Information Processing Society of Japan 1
2 n-best n-best n-best 1-best n-best 2. n-gram n-gram n-gram n-gram n-gram n-gram ( 1 ) n-gram ( 2 ) n-gram 1-best 1 ( 3 ) n-gram ( 4 ) n-gram 2 1 3. n-gram n-gram Ja En(Ref) kyo-chan -lrb- city bus -rrb- En(MT) kyoto chan -lrb- kyoto city bus -rrb- Rules SYMP ( x0:sym SYMP ( NP ( NN ( ) NN ( ) ) x1:sym ) ) x0 kyoto city bus x1 Eval 1 4 5 n-gram 3 n-gram (x) p(x, c = e) p(x, c = c) p(x) p( x, c = e) p( x, c = c) p( x) p(c = e) p(c = c) 1.0 1-best E k C k n-gram n-gram R k Ek n-best R k EV ( BLEU+1[5]) EV n-gram 1-best 3.1 n-gram F k F E k C k C k n-gram x E k # k (x) e k (x) # k (x) x / C k e k (x) = 0 (otherwise) e k ϕ e (x) = k e k(x) ϕ e n-gram 3.2 n-gram n-gram x n-gram c 2012 Information Processing Society of Japan 2
PMI PMI [1] P MI(x) = log p(x, c = e) p(x) p(c = e) p 3 p(x) n-gram x p(c = e) n-gram p(x, c = e) x PMI S(x) = p(x, c = e) log p(x, c = e) p(x) p(c = e) (1) n-gram (1) S(x) 3.3 n-gram n-gram F k F E k C k n-gram E k e k (x) C k c k (x) ϕ e(x) = k e k (x) ϕ c = k c k (x) n-gram n-gram p(c = e w = x) = ϕ e(x) ϕ e(x) + ϕ c(x) 1 n-gram 1 MacKay [6] n-gram x (2) S(x) n-gram S(x) = ϕ e(x) + αp e ϕ e(x) + ϕ c(x) + α x P e = ϕ e(x) x ϕ e(x) + x ϕ c(x) (2) α n-gram [13] (3) P = x c {e,c} #(w=x,c) k=0 (k + αp (c)) #(w=x) k=0 (k + α) (3) (3) P α P α P 4. n-gram [16] n-gram F = {F 1,..., F K } n-best Ê = {Ê 1,..., Ê K } R = {R 1,..., R K } Ê k = {Êk,1, Êk,2,..., Êk,I} ϕ(êk,i) w ϕ(êk,i) w n-best E k w 4.1 w [2] Ek Êk ϕ(êk) ϕ(ek ) w Ek Êk 0 F N 4.2 L1 [14] [12] L1 w L1 w 1 = i w i 0 L1 (FOBOS) [3] FOBOS 4.3 : ( 1 ) ϕ s : n-best ( 2 ) n-gram ϕ n : n-gram n-gram ( 3 ) ϕ l : c 2012 Information Processing Society of Japan 3
3 KFTT Train 330k 5.91M 6.09M Dev 1166 24.3k 26.8k Test 1160 26.7k 28.5k Ja En(Ref) En(MT) Eval he was the 1st seii taishogun of the muromachi shogunate. he was the first seii taishogun of the muromachi bakufu -lrb- japanese feudal government headed by a shogun -rrb-. 2 n-gram 4.4 n-best 1-best 1-best Êk R k {Êk, R k } n-best ϕ s 1-best 0 1 5. 2 n-gram x n-gram (4) W (x) = #(x, state = error) #(x) (4) #(x, state = error) n-gram #(x) n-gram n-gram 5.1 (KFTT)[7] 3 Travatar [8] Forest-to-String Nile *1 Egret *2 MERT [9] BLEU[10] 3 4 *1 http://code.google.com/p/nile/ *2 http://code.google.com/p/egret-parser/ ( 1 ) ( 2 ) ( 3 ) ( 4 ) ( 5 ) 2 n-best 100 10000 n-best 4.2 FOBOS L1 10 7-10 2 KFTT BLEU+1 100 n-gram 1-gram 3-gram n-best BLEU+1 RIBES[4] 2 n-gram n-gram n-gram 1 1 n-gram n-gram 1 n-gram 2 bakufu -lrb- japanese japanese feudal government 1 2 n-gram bakufu -lrb- japanese feudal government 1 n-gram 5.2 n-gram 2 30 30 n-gram c 2012 Information Processing Society of Japan 4
4 n-gram 3 3 oracle ref oracle ref oracle ref n-best ref 0.483 0.290 0.323 0.427 0.410 0.607 0.713 0.598 0.387 0.124 0.460 0.258 0.500 0.520 0.218 0.449 0.332 0.259 0.166 0.023 0.052 0.086 0.049 0.016 0.140 0.067 0.078 0.111 0.195 0.278 0.078 0.260 0.537 0.364 0.164 0.276 0.062 0.103 0.093 0.016 0.041 0.042 0.023 0.245 0.052 0.319 0.218 0.299 0.305 0.106 0.176 0.023 0.192 0.155 0.093 0 0.021 0.016 0.024 0.011 0 0 0.181 4 n-best 3 n-best 2 n-best n-gram n-best n-best n-gram n-best 1-best n-best 1-best n-best n-gram 5.3 n-gram n-gram 5 1-best 5 n-gram dev test 1.1 0.2 oracle 381.0 402.8 ref 368.0 378.6 oracle 135.4 151.9 ref 225.1 225.1 oracle 21.4 11.2 ref 19.7 7.7 n-best 6.2 4.0 ref 175.9 183.1 n-gram 4 n-gram n-best n-gram n-best n-gram n-gram 1-best 1-best n-gram n-gram 5.4 n-best n-gram BLEU RIBES 2 n-gram 6 BLEU RIBES BLEU RIBES c 2012 Information Processing Society of Japan 5
6 +BLEU +RIBES 0.598 0.529 0.332 0.216 0.067 0 0.164 0.285 0.245 0.125 0.192 0.289 0 0.086 RIBES BLEU 6. n-gram n-best n-gram n-best recall n-best 3 [7] Neubig, G.: The Kyoto Free Translation Task, http://www.phontron.com/kftt (2011). [8] Neubig, G.: Travatar: A Forest-to-String Machine Translation Engine based on Tree Transducers, Proc. ACL (2013). [9] Och, F. J.: Minimum Error Rate Training in Statistical Machine Translation, Proc. ACL (2003). [10] Papineni, K., Roukos, S., Ward, T. and Zhu, W.-J.: BLEU: a method for automatic evaluation of machine translation, Proc. ACL, pp. 311 318 (2002). [11] Popovic, M. and Ney, H.: Towards automatic error analysis of machine translation output, Computational Linguistics, pp. 657 688 (2011). [12] Roark, B., Saraclar, M. and Collins, M.: Discriminative n-gram language modeling, Computer Speech & Language, Vol. 21, No. 2, pp. 373 392 (2007). [13] Teh, Y. W., Jordan, M. I., Beal, M. J. and Blei, D. M.: Hierarchical Dirichlet processes, Journal of the American Statistical Association, Vol. 101, No. 476 (2006). [14] Tsuruoka, Y., Tsujii, J. and Ananiadou, S.: Stochastic Gradient Descent Training for L1-regularized Log-linear Models with Cumulative Penalty, Proc. ACL, pp. 477 485 (2009). [15] Vilar, D., Xu, J., D Haro, L. F. and Ney, H.: Error analysis of statistical machine translation output, Proc. LREC, pp. 697 702 (2006). [16] Neubig, G. Sakti, S. 20 (NLP2014) pp. 959-962 (2014). [1] Church, K. W. and Hank, P.: Word association norms, mutual information, and lexicography, Computational Linguistics, Vol. 10, pp. 22 29 (1990). [2] Collins, M.: Discriminative Training Methods for Hidden Markov Models: Theory and Experiments with Perceptron Algorithms, Proc. EMNLP, pp. 1 8 (2002). [3] Duchi, J. and Singer, Y.: Efficient Online and Batch Learning using Forward Backward Splitting, Journal of Machine Learning Research, Vol. 10 (2009). [4] Isozaki, H., Hirao, T., Duh, K., Sudoh, K. and Tsukada, H.: Automatic Evaluation of Translation Quality for Distant Language Pairs, Proc. EMNLP, pp. 944 952 (2010). [5] Lin, C.-Y. and Och, F. J.: Orange: a method for evaluating automatic evaluation metrics for machine translation, Proc. COLING, pp. 501 507 (2004). [6] Mackay, D. J. and Petoy, L. C. B.: A Hierarchical Dirichlet Language Model, Natural Language Engineering, Vol. 1 (1995). c 2012 Information Processing Society of Japan 6