- Pedram Hosseini, David A. Broniatowski, Mona T. Diab. 2021. Commonsense Knowledge-Augmented Pretrained Language Models for Causal Reasoning Classification. Abstract: Commonsense knowledge can be leveraged 001 for identifying causal relations in text. In this 002 work, we convert triples in ATOMIC 2020 , a wide 003 coverage commonsense reasoning knowledge 004 graph, to natural language text and continually 005 pretrain a BERT pretrained language model. 006 We evaluate the resulting model on answer- 007 ing commonsense reasoning questions. Our 008 results show that a continually pretrained lan- 009 guage model augmented with commonsense 010 reasoning knowledge outperforms our base- 011 line on two commonsense causal reasoning 012 benchmarks, COPA and BCOPA-CE, without 013 additional improvement on the base model or 014 using quality-enhanced data for ﬁne-tuning.
