@inproceedings{cao-etal-2023-improving,
    title = "Improving Autoregressive Grammatical Error Correction with Non-autoregressive Models",
    author = "Cao, Hang  and
      Cao, Zhiquan  and
      Hu, Chi  and
      Hou, Baoyu  and
      Xiao, Tong  and
      Zhu, Jingbo",
    editor = "Rogers, Anna  and
      Boyd-Graber, Jordan  and
      Okazaki, Naoaki",
    booktitle = "Findings of the Association for Computational Linguistics: ACL 2023",
    month = jul,
    year = "2023",
    address = "Toronto, Canada",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.findings-acl.760",
    doi = "10.18653/v1/2023.findings-acl.760",
    pages = "12014--12027",
    abstract = "Grammatical Error Correction (GEC) aims to correct grammatical errors in sentences. We find that autoregressive models tend to assign low probabilities to tokens that need corrections. Here we introduce additional signals to the training of GEC models so that these systems can learn to better predict at ambiguous positions. To do this, we use a non-autoregressive model as an auxiliary model, and develop a new regularization term of training by considering the difference in predictions between the autoregressive and non-autoregressive models. We experiment with this method on both English and Chinese GEC tasks. Experimental results show that our GEC system outperforms the baselines on all the data sets significantly.",
}

Grammatical Error Correction (GEC) aims to correct grammatical errors in sentences. We find that autoregressive models tend to assign low probabilities to tokens that need corrections. Here we introduce additional signals to the training of GEC models so that these systems can learn to better predict at ambiguous positions. To do this, we use a non-autoregressive model as an auxiliary model, and develop a new regularization term of training by considering the difference in predictions between the autoregressive and non-autoregressive models. We experiment with this method on both English and Chinese GEC tasks. Experimental results show that our GEC system outperforms the baselines on all the data sets significantly.
Cao, Hang Cao, Zhiquan Hu, Chi Hou, Baoyu Xiao, Tong Zhu, Jingbo
Improving Autoregressive Grammatical Error Correction with Non-autoregressive Models