@inproceedings{deoghare-etal-2023-multi,
    title = "A Multi-task Learning Framework for Quality Estimation",
    author = "Deoghare, Sourabh  and
      Choudhary, Paramveer  and
      Kanojia, Diptesh  and
      Ranasinghe, Tharindu  and
      Bhattacharyya, Pushpak  and
      Or{\u{a}}san, Constantin",
    editor = "Rogers, Anna  and
      Boyd-Graber, Jordan  and
      Okazaki, Naoaki",
    booktitle = "Findings of the Association for Computational Linguistics: ACL 2023",
    month = jul,
    year = "2023",
    address = "Toronto, Canada",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.findings-acl.585",
    doi = "10.18653/v1/2023.findings-acl.585",
    pages = "9191--9205",
    abstract = "Quality Estimation (QE) is the task of evaluating machine translation output in the absence of reference translation. Conventional approaches to QE involve training separate models at different levels of granularity viz., word-level, sentence-level, and document-level, which sometimes lead to inconsistent predictions for the same input. To overcome this limitation, we focus on jointly training a single model for sentence-level and word-level QE tasks in a multi-task learning framework. Using two multi-task learning-based QE approaches, we show that multi-task learning improves the performance of both tasks. We evaluate these approaches by performing experiments in different settings, viz., single-pair, multi-pair, and zero-shot. We compare the multi-task learning-based approach with baseline QE models trained on single tasks and observe an improvement of up to 4.28{\%} in Pearson{'}s correlation (r) at sentence-level and 8.46{\%} in F1-score at word-level, in the single-pair setting. In the multi-pair setting, we observe improvements of up to 3.04{\%} at sentence-level and 13.74{\%} at word-level; while in the zero-shot setting, we also observe improvements of up to 5.26{\%} and 3.05{\%}, respectively. We make the models proposed in this paper publically available.",
}

Quality Estimation (QE) is the task of evaluating machine translation output in the absence of reference translation. Conventional approaches to QE involve training separate models at different levels of granularity viz., word-level, sentence-level, and document-level, which sometimes lead to inconsistent predictions for the same input. To overcome this limitation, we focus on jointly training a single model for sentence-level and word-level QE tasks in a multi-task learning framework. Using two multi-task learning-based QE approaches, we show that multi-task learning improves the performance of both tasks. We evaluate these approaches by performing experiments in different settings, viz., single-pair, multi-pair, and zero-shot. We compare the multi-task learning-based approach with baseline QE models trained on single tasks and observe an improvement of up to 4.28{\%} in Pearson{'}s correlation (r) at sentence-level and 8.46{\%} in F1-score at word-level, in the single-pair setting. In the multi-pair setting, we observe improvements of up to 3.04{\%} at sentence-level and 13.74{\%} at word-level; while in the zero-shot setting, we also observe improvements of up to 5.26{\%} and 3.05{\%}, respectively. We make the models proposed in this paper publically available.
Deoghare, Sourabh Choudhary, Paramveer Kanojia, Diptesh Ranasinghe, Tharindu Bhattacharyya, Pushpak Or{\u{a}}san, Constantin
A Multi-task Learning Framework for Quality Estimation