An HMM approach to vowel restoration in Arabic and Hebrew

Citations Over TimeTop 14% of 2002 papers

Abstract

Semitic languages pose a problem to Natural Language Processing since most of the vowels are omitted from written prose, resulting in considerable ambiguity at the word level. However, while reading text, native speakers can generally vocalize each word based on their familiarity with the lexicon and the context of the word. Methods for vowel restoration in previous work involving morphological analysis concentrated on a single language and relied on a parsed corpus that is difficult to create for many Semitic languages. We show that Hidden Markov Models are a useful tool for the task of vowel restoration in Semitic languages. Our technique is simple to implement, does not require any language specific knowledge to be embedded in the model and generalizes well to both Hebrew and Arabic. Using a publicly available version of the Bible and the Qur'an as corpora, we achieve a success rate of 86% for restoring the exact vowel pattern in Arabic and 81% in Hebrew. For Hebrew, we also report on 87% success rate for restoring the correct phonetic value of the words.

Related Papers

• → The role of vowels in reading Semitic scripts: Data from Arabic and Hebrew(2001)212 cited
• → Acoustic Measurements of the ‘Fake’ Vowel Length and Degrees of Vowel Length in St. Lawrence Island Eskimo(1974)4 cited
• The Narrative Role of Semitic Languages in the Book(2003)
• → Acoustic Measurements of the “Fake” Vowel Length and the Three Degrees of Vowel Length in Yupik Eskimo(1973)
• → Covert vowel length in Korean(2017)