1530 ( ) 2014,54(12),, E (, 1, X ) [4],,, α, T α, β,, T β, c, P(T β 1 T α,α, β,c) 1 1,,X X F, X E F X E X F X F E X E 1 [1-2] , 2 : X X 1 X 2 ;

Μέγεθος: px

Εμφάνιση ξεκινά από τη σελίδα:

Download "1530 ( ) 2014,54(12),, E (, 1, X ) [4],,, α, T α, β,, T β, c, P(T β 1 T α,α, β,c) 1 1,,X X F, X E F X E X F X F E X E 1 [1-2] , 2 : X X 1 X 2 ;"

Ζαχαρίας Μπουκουβαλαίοι
5 χρόνια πριν
Προβολές:

1 ISSN CN /N ( ) JTsinghuaUniv(Sci& Technol), 2014,Vol.54, No.12 4/ ,, (,, (), ) [1-2] :,,,,,,,, :, 0.3~ [3] 0.8BLEU,, : ; ; [4], ; :TP391.2 :A, : (2014) , [5-8] Neuralreorderingmodelforhierarchical phrase-basedtranslations LIPeng,LIUYang,SUNMaosong (StateKeyLaboratoryofInteligentTechnologyandSystems, TsinghuaNationalLaboratoryforInformationScienceand Technology,DepartmentofComputerScienceandTechnology, TsinghuaUniversity,Beijing100084,China) Abstract:Thereorderingambiguityisoneofthemajorchalengesfor hierarchicalphrase-basedtranslation models.these models only considerlimitedcontextssothattheirabilityisreducedtoresolve reorderingambiguities.morecontexts wereintroducedintothese modelsusinganeuralreorderingmodelforhierarchicalphrase-based translations.reorderingistreatedasaclassificationprobleminthis model.thevector-spacerepresentationsarecomputedforphrases usingrecursiveauto-encoders.theserepresentationsarethenused asfeatures to predict the probabilities of various reorderings. Finaly,these probabilities are used as new features for the decoding.testsshowthatthismodelimprovesthebleuscoreby overthebaselinesforChinese-Englishtranslation,which indicatesthatthismodelgivesbeterreorderingthanthebaselines. Keywords:computer science and technology; neural network; reordering model;recursiveauto-encoders;hierarchical phrase-basedtranslation,, [3-4,6],, [911],; (recursiveautoencoder) [11],, : : (2012AA011102); ( ); (2014BAK101303) : (1987 ),(),, :,, liuyang2011@tsinghua.edu.cn

2 1530 ( ) 2014,54(12),, E (, 1, X ) [4],,, α, T α, β,, T β, c, P(T β 1 T α,α, β,c) 1 1,,X X F, X E F X E X F X F E X E 1 [1-2] , 2 : X X 1 X 2 ;X 1X 2. [911] [11],, 2 ω1 ω2, c 1 c 2, (1) () X, p ω1ω2 : X 1 X 2 X 1X 2 X 1 p =f (1) (W (1) [c 1 ;c 2 ]+b (1) ). (1) X 2 X 1X 2 X, :W ;b (1) ;[c 1 ;c 2 ] c 1 c 2 ;f (1) ( ), 2 X 1 beautiful X X 1 X 2 ;X 1 X 2 tanh( ) c 1 c 2 c 1 c 2, (2) : +X beautiful [c 1 ;c 2 ]=f (2) (W (2) p+b (2) ). (2) X X 2 ;beautifulx 2 :W (2) ;b (2) ;f (2) ( ), tanh( ) c 1 c 1 c 2, c 2 ( ), p X X 1 X 2 ;X 2ofX 1 c 1 c 2, (3) c 1 c 1 c 2 c 2 :, 1, 2 ( c 1 -c c 2 -c 2 2 ). (3), 2, [11], 2.1 2, 2, [4], 1 1 [11] 1, X, F

3 ,: X F X F 2.3 2) : (cross-entropyerror) 1, 1 [11], X F X F, 6 L-BFGS [12], (backpropagationthroughstruc- F ( ) tures) [13],2 2 X ( ),2 X, X 2 X ( Bolivia presidentialandparliament ),,, (4), softmax(w o T α p o +b o T α ). (4) :W o T α b o T α T α 4,P o 4.1 ( 1 6 ) 123,, 0.32, 0.35, 4, GIGAWORD Xinhua 1X,4 (LDC2011T07), 3.986, ( 1), 4, SRILM [14] NIST2006 2X,14,NIST 2003-, 14,W o T α b o T α , 2 : ( MT03~05 ) BLEU [15] 1) : 2 : [1-2] [4],,

4 1532 ( ) 2014,54(12), [4],, MERT, [16] ,, 2,,, (, ), 2.12%,, 3, MT06 3,2, MERT 1X, 2X, 4, 14, 2 BLEU X F X, MT06 MT03 MT04 MT , 91.40% X E X, , , 3 /% /% X F X F X F X F X F X F X F X X F X F F X F X F X X X1 X X1 X X1 X X1 X (References),, Linguistics,2005: [1] Chiang D.A hierarchicalphrase-based modelforstatistical machinetranslation [C]// Proceedingsofthe43rd Annual Meeting on Association for Computational Linguistics. Stroudsburg, PA, USA: Association for Computational [2] Chiang D. Hierarchical phrase-based translation [J]., ComputationalLinguistics,2007,33(2):

5 ,: 1533 [3] He Z, Liu Q, Lin S. Improving statistical machine translationusinglexicalizedruleselection [C]//Proceedings of the 22nd International Conference on Computational Linguistics. Manchester, UK: Coling 2008 Organizing Commitee,2008: [4] HeZ, Meng Y,Yu H. Maximum entropy based phrase reorderingfor hierarchicalphrase-based translation [C]// Proceedingsofthe2010ConferenceonEmpiricalMethodsin Natural Language Processing. Massachusets, USA: AssociationforComputationalLinguistics,2010: [5] Zens R, Ney H. Discriminative reordering models for statistical machine translation [C]// Proceedings on the Workshopon Statistical Machine Translation.New York, USA: Association for Computational Linguistics, 2006: [6] XiongD,Liu Q,Lin S. Maximum entropy based phrase reordering modelforstatistical machinetranslation [C]// Proceedings of the 21st International Conference on ComputationalLinguisticsandthe44thannualmeetingofthe Association for Computational Linguistics. Sydney, Australia: Association for Computational Linguistics, 2006: [7] XiongD,Zhang M,Aw A,etal.Linguisticalyannotated BTGforstatisticalmachinetranslation [C]//Proceedingsof the 22nd International Conference on Computational Linguistics.Manchester,UK:AssociationforComputational Linguistics,2008: [8] LiuQ,HeZ,Liu Y,etal.Maximum entropybasedrule selection model for syntax-based statistical machine translation [C]// Proceedings ofthe 2008 Conference on Empirical Methods in Natural Language Processing. Honolulu, Hawai, USA: Association for Computational Linguistics,2008: [9] Bengio Y, Ducharme R, Vincent P, et al. A neural probabilistic language model [J]. Journal of Machine Learning Research,2003,3: [10]ColobertR,WestonJ,Botou L,etal.Naturallanguage processing (almost)from scratch [J].Journalof Machine Learning Research,2011,12: [11]SocherR,PenningtonJ,HuangE H,etal.Semi-supervised recursiveautoencodersforpredictingsentimentdistributions [C]// Proceedings ofthe 2011 Conference on Empirical Methods in Natural Language Processing. Edinburgh, Scotland,UK:Associationfor ComputationalLinguistics, 2011: [12]LiuD C,NocedalJ.Onthelimited memorybfgs method for large scale optimization [J]. Mathematical Programming,1989,45(1-3): [13]GolerC,Kuchler A.Learningtask-dependentdistributed representationsbybackpropagationthroughstructure [C]// Proceedings of the International Conference on Neural Networks(ICNN 96).WashingtonDC,USA:IEEE,1996: [14]StolckeA.SRILM-anextensiblelanguage modelingtoolkit [C]//ProceedingsoftheInternationalConferenceonSpoken Language Processing. Denver, Colorado, USA: ISCA, 2002: [15]PapineniK,RoukosS,WardT,etal.BLEU:A methodfor automatic evaluation of machine translation [C]// Proceedingsofthe40th Annual Meetingon Associationfor Computational Linguistics. Philadelphia, Pennsylvania, USA: Association for Computational Linguistics, 2002: [16]OchFJ.Minimumerrorratetraininginstatisticalmachine translation [C]//Proceedingsofthe41stAnnualMeetingon AssociationforComputationalLinguistics.Sapporo,Japan: AssociationforComputationalLinguistics,2003:

Σχετικά έγγραφα

1 n-gram n-gram n-gram [11], [15] n-best [16] n-gram. n-gram. 1,a) Graham Neubig 1,b) Sakriani Sakti 1,c) 1,d) 1,e)

1 n-gram n-gram n-gram [11], [15] n-best [16] n-gram. n-gram. 1,a) Graham Neubig 1,b) Sakriani Sakti 1,c) 1,d) 1,e) 1,a) Graham Neubig 1,b) Sakriani Sakti 1,c) 1,d) 1,e) 1. [11], [15] 1 Nara Institute of Science and Technology a) akabe.koichi.zx8@is.naist.jp b) neubig@is.naist.jp c) ssakti@is.naist.jp d) tomoki@is.naist.jp