ISSN1000-0054 CN11-2223/N ( ) 2014 54 12 JTsinghuaUniv(Sci& Technol), 2014,Vol.54, No.12 4/20 1529-1533,, (,, (), 100084) [1-2] :,,,,,,,, :, 0.3~ [3] 0.8BLEU,, : ; ; [4], ; :TP391.2 :A, :1000-0054(2014)12-1529-05, [5-8] Neuralreorderingmodelforhierarchical phrase-basedtranslations LIPeng,LIUYang,SUNMaosong (StateKeyLaboratoryofInteligentTechnologyandSystems, TsinghuaNationalLaboratoryforInformationScienceand Technology,DepartmentofComputerScienceandTechnology, TsinghuaUniversity,Beijing100084,China) Abstract:Thereorderingambiguityisoneofthemajorchalengesfor hierarchicalphrase-basedtranslation models.these models only considerlimitedcontextssothattheirabilityisreducedtoresolve reorderingambiguities.morecontexts wereintroducedintothese modelsusinganeuralreorderingmodelforhierarchicalphrase-based translations.reorderingistreatedasaclassificationprobleminthis model.thevector-spacerepresentationsarecomputedforphrases usingrecursiveauto-encoders.theserepresentationsarethenused asfeatures to predict the probabilities of various reorderings. Finaly,these probabilities are used as new features for the decoding.testsshowthatthismodelimprovesthebleuscoreby 0.3 0.8overthebaselinesforChinese-Englishtranslation,which indicatesthatthismodelgivesbeterreorderingthanthebaselines. Keywords:computer science and technology; neural network; reordering model;recursiveauto-encoders;hierarchical phrase-basedtranslation,, [3-4,6],, [911],; (recursiveautoencoder) [11],, :2014-09-22 : (2012AA011102); (61331013); (2014BAK101303) : (1987 ),(),, :,,E-mail:liuyang2011@tsinghua.edu.cn
1530 ( ) 2014,54(12),, E (, 1, X ) [4],,, α, T α, β,, T β, c, P(T β 1 T α,α, β,c) 1 1,,X X F, X E F X E X F X F E X E 1 [1-2] 2.2 1 2, 2 : X X 1 X 2 ;X 1X 2. [911] [11],, 2 ω1 ω2, c 1 c 2, (1) () X, p ω1ω2 : X 1 X 2 X 1X 2 X 1 p =f (1) (W (1) [c 1 ;c 2 ]+b (1) ). (1) X 2 X 1X 2 X, :W ;b (1) ;[c 1 ;c 2 ] c 1 c 2 ;f (1) ( ), 2 X 1 beautiful X X 1 X 2 ;X 1 X 2 tanh( ) c 1 c 2 c 1 c 2, (2) : +X beautiful [c 1 ;c 2 ]=f (2) (W (2) p+b (2) ). (2) X X 2 ;beautifulx 2 :W (2) ;b (2) ;f (2) ( ), tanh( ) c 1 c 1 c 2, c 2 ( ), p X X 1 X 2 ;X 2ofX 1 c 1 c 2, (3) c 1 c 1 c 2 c 2 :, 1, 2 ( c 1 -c 1 2 + c 2 -c 2 2 ). (3), 2, [11], 2.1 2, 2, [4], 1 1 [11] 1, X, F
,: 1531 1 X F X F 2.3 2) : (cross-entropyerror) 1, 1 [11], X F X F, 6 L-BFGS [12], 6 2 2 (backpropagationthroughstruc- F ( ) tures) [13],2 2 X ( ),2 X, X 2 X ( Bolivia presidentialandparliament ),,, (4), softmax(w o T α p o +b o T α ). (4) :W o T α b o T α T α 4,P o 4.1 ( 1 6 ) 123,, 0.32, 0.35, 4, GIGAWORD Xinhua 1X,4 (LDC2011T07), 3.986, ( 1), 4, SRILM [14] NIST2006 2X,14,NIST 2003-, 14,W o T α b o T α 2005 3, 2 : ( MT03~05 ) BLEU [15] 1) : 2 : [1-2] [4],,
1532 ( ) 2014,54(12), [4],, MERT, [16] 4.2 200 3,, 2,,, (, ), 2.12%,, 3, MT06 3,2, MERT 1X, 2X, 4, 14, 2 BLEU X F X, MT06 MT03 MT04 MT05 33638939, 91.40% 31.64 33.18 33.98 31.77 X E X, 31.93 33.52 33.95 31.81, 31.81 33.81 34.50 32.61, 3 /% /% X F X 20.36 69.67 77.00 F X F X F X F X F X F X X F X F F X F X F X 20.40 64.67 81.83 X 15.88 74.83 81.50 X1 X2 20.43 46.50 58.89 X1 X2 10.41 48.96 53.86 X1 X2 10.40 48.14 52.14 X1 X2 2.12 36.36 33.43 5 (References),, Linguistics,2005:263-270. [1] Chiang D.A hierarchicalphrase-based modelforstatistical machinetranslation [C]// Proceedingsofthe43rd Annual Meeting on Association for Computational Linguistics. Stroudsburg, PA, USA: Association for Computational [2] Chiang D. Hierarchical phrase-based translation [J]., ComputationalLinguistics,2007,33(2):201-228.
,: 1533 [3] He Z, Liu Q, Lin S. Improving statistical machine translationusinglexicalizedruleselection [C]//Proceedings of the 22nd International Conference on Computational Linguistics. Manchester, UK: Coling 2008 Organizing Commitee,2008:321-328. [4] HeZ, Meng Y,Yu H. Maximum entropy based phrase reorderingfor hierarchicalphrase-based translation [C]// Proceedingsofthe2010ConferenceonEmpiricalMethodsin Natural Language Processing. Massachusets, USA: AssociationforComputationalLinguistics,2010:555-563. [5] Zens R, Ney H. Discriminative reordering models for statistical machine translation [C]// Proceedings on the Workshopon Statistical Machine Translation.New York, USA: Association for Computational Linguistics, 2006: 55-63. [6] XiongD,Liu Q,Lin S. Maximum entropy based phrase reordering modelforstatistical machinetranslation [C]// Proceedings of the 21st International Conference on ComputationalLinguisticsandthe44thannualmeetingofthe Association for Computational Linguistics. Sydney, Australia: Association for Computational Linguistics, 2006:521-528. [7] XiongD,Zhang M,Aw A,etal.Linguisticalyannotated BTGforstatisticalmachinetranslation [C]//Proceedingsof the 22nd International Conference on Computational Linguistics.Manchester,UK:AssociationforComputational Linguistics,2008:1009-1016. [8] LiuQ,HeZ,Liu Y,etal.Maximum entropybasedrule selection model for syntax-based statistical machine translation [C]// Proceedings ofthe 2008 Conference on Empirical Methods in Natural Language Processing. Honolulu, Hawai, USA: Association for Computational Linguistics,2008:89-97. [9] Bengio Y, Ducharme R, Vincent P, et al. A neural probabilistic language model [J]. Journal of Machine Learning Research,2003,3:1137-1155. [10]ColobertR,WestonJ,Botou L,etal.Naturallanguage processing (almost)from scratch [J].Journalof Machine Learning Research,2011,12:2493-2537. [11]SocherR,PenningtonJ,HuangE H,etal.Semi-supervised recursiveautoencodersforpredictingsentimentdistributions [C]// Proceedings ofthe 2011 Conference on Empirical Methods in Natural Language Processing. Edinburgh, Scotland,UK:Associationfor ComputationalLinguistics, 2011:151-161. [12]LiuD C,NocedalJ.Onthelimited memorybfgs method for large scale optimization [J]. Mathematical Programming,1989,45(1-3):503-528. [13]GolerC,Kuchler A.Learningtask-dependentdistributed representationsbybackpropagationthroughstructure [C]// Proceedings of the International Conference on Neural Networks(ICNN 96).WashingtonDC,USA:IEEE,1996: 347-352. [14]StolckeA.SRILM-anextensiblelanguage modelingtoolkit [C]//ProceedingsoftheInternationalConferenceonSpoken Language Processing. Denver, Colorado, USA: ISCA, 2002:901-904. [15]PapineniK,RoukosS,WardT,etal.BLEU:A methodfor automatic evaluation of machine translation [C]// Proceedingsofthe40th Annual Meetingon Associationfor Computational Linguistics. Philadelphia, Pennsylvania, USA: Association for Computational Linguistics, 2002:311-318. [16]OchFJ.Minimumerrorratetraininginstatisticalmachine translation [C]//Proceedingsofthe41stAnnualMeetingon AssociationforComputationalLinguistics.Sapporo,Japan: AssociationforComputationalLinguistics,2003:160-167.