Measuring Human Readability of Machine Generated Text: Three Case Studies in Speech Recognition and Machine Translation

Citations Over TimeTop 10% of 2006 papers

Abstract

We present highlights from three experiments that test the readability of current state-of-the art system output from: (1) an automated English speech-to-text (SST) system; (2) a text-based Arabic-to-English machine translation (MT) system; and (3) an audio-based Arabic-to-English MT process. We measure readability in terms of reaction time and passage comprehension in each case, applying standard psycholinguistic testing procedures and a modified version of the standard defense language proficiency test for Arabic called the DLPT*. We learned that: (1) subjects are slowed down by about 25% when reading system STT output; (2) text-based MT systems enable an English speaker to pass Arabic Level 2 on the DLPT*; and (3) audio-based MT systems do not enable English speakers to pass Arabic Level 2. We intend for these generic measures of readability to predict performance of more application-specific tasks.

Related Papers

• → Evaluation of the readability of ACOG patient education pamphlets(1999)44 cited
• The Validity of Some Popular Readability Formulas(2012)
• → The readability of advertisements and articles in trade journals(1990)18 cited
• → Evaluation of the Readability of ACOG Patient Education Pamphlets(1999)7 cited
• A Survey of Studies on Readability(2000)