A hybrid approach to compiling bilingual dictionaries of medical terms from parallel corpora

Transcript

1 A hybrid approach to compiling bilingual dictionaries of medical terms from parallel corpora Georgios Kontonatsios 14 th October /10/2014 1

2 Overview Background Parallel Corpus Problem Motivation Methods Random Forest Classifier Statistical Phrase Alignment Hybrid Approach Experiments English-Greek & English-Romanian Error Analysis Conclusions Discussion Future Work 2

3 Background: Parallel Corpus A parallel corpus is a collection of documents in a source language paired with their direct translation in a target language English Abraxane monotherapy is indicated for the treatment of metastatic breast cancer Greek η µονοθεραπεία µε abraxane ενδείκνυται για τη θεραπεία µεταστατικού καρκίνου µαστού 3

4 Background: Parallel Corpus 1) Useful for SMT 2) Relatively scarce resources Koehn (2005) trained 110 SMT systems (11 languages) in three weeks. Available finance, law, medicine etc. 3) Excellent resources for mining bilingual terminologies Exact translations => No missing translations of terms sentence aligned => limited search space of candidate translations Same size => term frequencies are comparable 4

5 Background: Problem Parallel Corpus Term Alignment Dictionary of MWT Abraxane monotherapy is indicated for the treatment of metastatic breast cancer η µονοθεραπεία µε abraxane ενδείκνυται για τη θεραπεία µεταστατικού καρκίνου µαστού metastatic breast cancer µεταστατικού καρκίνου µαστού 5

6 %Coverage of English UMLS Background: Biomedical Domain Existing resources in the biomedical domain remain incomplete UMLS A multilingual terminological resource (more than 20 languages) Indexes ~7.6M English terms 18.00% 16.00% 14.00% 12.00% ~6.3M missing tranlsations 16.44% 10.00% 8.00% 6.00% expand UMLS for English-Greek and English-Romanian 4.00% 2.00% 0.00% 1.72% 2.88% 2.59% 2.43% 1.21% 1.79% 3.26% 0.55% 2.06% 1.40% 7

7 Methodology: Term Alignment Pipeline Parallel Corpus MetaMap Term Alignment Link to UMLS Abraxane monotherapy is indicated for the treatment of metastatic breast cancer C , Neoplastic Process η µονοθεραπεία µε abraxane ενδείκνυται για τη θεραπεία µεταστατικού καρκίνου µαστού 8

8 Methodology: Term Alignment Algorithms Supervised machine learning method Random Forest Classifier (EACL 2014, EMNLP 2014) Exploits internal structure of terms (character n-gram feature representation) Requires positive and negative instances for training Out-of-domain seed dictionary (i.e. BabelNet) Unsupervised approach Part of Moses SMT (Koehn et al., 2007) Statistical Phrase Alignment (Koehn et al., 2003) (Out of the box solution) Exploits co-occurrences of source and target terms Works well for frequently occurring terms Performance decreases for rare terms 9

9 Methodology: Hybrid Approach For s to be translated, RF and SPA suggest N ranked candidate translations Classification margin Translation probability SPA 1) του σακχαρώδη διαβήτη τύπου 2 2) σακχαρώδη διαβήτη τύπου 2 3) σακχαρώδους διαβήτη τύπου 2 type 2 diabetes mellitus RF 1) διαβήτη τύπου 2 2) διαβήτη τύπου 2 και καρδιακή 3) σακχαρώδη διαβήτη τύπου 2 13

10 Methodology: Hybrid Approach Dictionaries containing N candidate translations have a limited number of applications (e.g., SMT) To enrich existing terminologies, human curators need to post-edit the output of term alignment methods Objective is to improve the precision of higher ranking candidates (precision@n=1) Intersection of RF and SPA; ranking candidates according to translation probability by SPA SPA 1) του σακχαρώδη διαβήτη τύπου 2 2) σακχαρώδη διαβήτη τύπου 2 3) σακχαρώδους διαβήτη τύπου 2 type 2 diabetes mellitus Voting RF 1) διαβήτη τύπου 2 2) διαβήτη τύπου 2 και καρδιακή 3) σακχαρώδη διαβήτη τύπου 2 1) σακχαρώδη διαβήτη τύπου 2 14

11 Experiments: Corpora EMEA (Tiedemann, 2009), a biomedical parallel corpus from European Medicines Agency - 1.5K sentence aligned documents in 22 languages - Drug usage guidelines en el en ro - 372K sentences - 17,907 unique English MWTs - 321K sentences - 16,625 unique English MWTs 15

12 Experiments: Evaluation Randomly sampled 1,000 English MWTs for each English MWT, we selected the top 20 translation candidates. en-el RF SPA Voting en-ro RF SPA Voting 16

13 Precision Experiments: Results RF SPA RF+SPA # candidate translations per source term English-Greek dataset 18

14 Precision Experiments: Results 1 RF SPA RF+SPA # candidate translations per source term English-Romanian dataset 19

15 Recall Experiments: Results RF SPA RF+SPA # candidate translations per source term English-Greek dataset 20

16 Recall Experiments: Results RF SPA RF+SPA # candidate translations per source term English-Romanian dataset 21

17 Error Analysis RF Partial matches urea cycle disorder discontinuous translations metabolic diseases (disorder) (cycle) (urea) διαταραχών του κύκλου της ουρίας (diseases) (hereditary) (metabolic) boli ereditare de metabolism SPA Statistically-based tool. -Performance largely affected by term frequency top-20 precision on terms having varying frequency 22

18 Top-20 Precision Error Analysis SPA RF RF + SPA Performance decreases for lower frequency terms [ ] [50 100] [25 50] [15 25] [10 15] [5 10] [1 5] frequency ranges English-Greek dataset 23

19 top-20 Precision Error Analysis SPA RF RF + SPA [ ] [50 100] [25 50] [15 25] [10 15] [5 10] [1 5] frequency ranges English-Romanian dataset 24

20 Discussion Hybrid approach Compilation of bilingual terminologies from parallel corpora Enrich UMLS with two under-resource languages Observations: Substantially improves top-1 precision of RF and SPA Outperforms SPA when translating low-frequency terms Low recall 25

21 Future Work Investigate integration of bilingual terminologies with SMT Parallel corpus SPA SPA RF Phrase table SMT LM Lower top-1 precision Poor performance for low-frequency terms 26

22 Questions? 27