- Houda Bouamor, Hanan Alshikhabobakr, Behrang Mohit, Kemal Oflazer. 2014. A Human Judgement Corpus and a Metric for Arabic MT Evaluation. Abstract: We present a human judgments datasetand an adapted metric for evaluation ofArabic machine translation. Our mediumscaledataset is the first of its kind for Arabicwith high annotation quality. We usethe dataset to adapt the BLEU score forArabic. Our score (AL-BLEU) providespartial credits for stem and morphologicalmatchings of hypothesis and referencewords. We evaluate BLEU, METEOR andAL-BLEU on our human judgments corpusand show that AL-BLEU has the highestcorrelation with human judgments. Weare releasing the dataset and software tothe research community.
