|
Automated Metrics for Speech Translation
December 2009
Sherri Condon, The MITRE Corporation
Mark Arehart, The MITRE Corporation
Christy Doran, The MITRE Corporation
Dan Parvaz, The MITRE Corporation
John Aberdeen, The MITRE Corporation
Karine Megerdoomian, The MITRE Corporation
Beatrice Oshika, The MITRE Corporation
Greg Sanders, National Institute of Standards and Technology
ABSTRACT
In this paper, we describe automated measures used to evaluate machine translation quality in the Defense Advanced Research Projects Agency's Spoken Language Communication and Translation System for Tactical Use program, which is developing speech translation systems for dialogue between English and Iraqi Arabic speakers in military contexts. Limitations of the automated measures are illustrated along with variants of the measures that seek to overcome those limitations. Both the dialogue structure of the data and the Iraqi Arabic language challenge these measures, and the paper presents some solutions adopted by MITRE and NIST to improve confidence in the scores.
Additional Search Keywords
n/a
|
