What are the different methods proposed for improving factual consistency in summaries generated by summarization systems?
Factual Consistency in Text Summarization
Factual consistency is an important quality of text
summarization systems (Kryscinski et al., 2020;
Maynez et al., 2020; Fabbri et al., 2021). Related
work has proposed various methods of improving
the factual consistency of summaries by (1) train-
ing abstractive summarization models with factu-
ality metrics (Goyal and Durrett, 2021; Cao et al.,
2022), (2) introducing new training objectives and multi-task learning for model training (Cao and
Wang, 2021; Zhu et al., 2021; Aralikatte et al.,
2021; Zhang et al., 2022; Xu and Zhao, 2022), (3)
post-editing or re-ranking the initially generated
summaries to improve the factual consistency (Cao
et al., 2020; Chen et al., 2021; Balachandran et al.,
2022; Fabbri et al., 2022b), (4) designing factuality-
aware pre-training (Wan and Bansal, 2022)
Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling
Abstractive summarization models often generate inconsistent summaries containing factual errors or hallucinated content. Recent works focus on correcting factual errors in generated summaries via post-editing. Such correction models are trained using adversarial non-factual summaries constructed using heuristic rules for injecting errors. However, generating non-factual summaries using heuristics often does not generalize well to actual model errors. In this work, we propose to generate hard, representative synthetic examples of non-factual summaries through infilling language models. With this data, we train a more robust fact-correction model to post-edit the summaries to improve factual consistency. Through quantitative and qualitative experiments on two popular summarization datasets— CNN/DM and XSum—we show that our approach vastly outperforms prior methods in correcting erroneous summaries. Our model—FactEdit—improves factuality scores by over ~11 points on CNN/DM and over ~31 points on XSum on average across multiple summarization models, producing more factual summaries while maintaining competitive summarization quality.
[{"first":"Vidhisha","last":"Balachandran","middle":[]},{"first":"Hannaneh","last":"Hajishirzi","middle":[]},{"first":"William","last":"Cohen","middle":[]},{"first":"Yulia","last":"Tsvetkov","middle":[]}]
Factual Error Correction for Abstractive Summarization Models
Neural abstractive summarization systems have achieved promising progress, thanks to the availability of large-scale datasets and models pre-trained with self-supervised methods. However, ensuring the factual consistency of the generated summaries for abstractive summarization systems is a challenge. We propose a post-editing corrector module to address this issue by identifying and correcting factual errors in generated summaries. The neural corrector model is pre-trained on artificial examples that are created by applying a series of heuristic transformations on reference summaries. These transformations are inspired by an error analysis of state-of-the-art summarization model outputs. Experimental results show that our model is able to correct factual errors in summaries generated by other neural summarization models and outperforms previous models on factual consistency evaluation on the CNN/DailyMail dataset. We also find that transferring from artificial error correction to downstream settings is still very challenging.
[{"first":"Meng","last":"Cao","middle":[]},{"first":"Yue","last":"Dong","middle":[]},{"first":"Jiapeng","last":"Wu","middle":[]},{"first":"Jackie","last":"Cheung","middle":["Chi","Kit"]}]
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
We present FactPEGASUS, an abstractive summarization model that addresses the problem of factuality during pre-training and fine-tuning: (1) We augment the sentence selection strategy of PEGASUS’s (Zhang et al., 2019) pre-training objective to create pseudo-summaries that are both important and factual; (2) We introduce three complementary components for fine-tuning. The corrector removes hallucinations present in the reference summary, the contrastor uses contrastive learning to better differentiate nonfactual summaries from factual ones, and the connector bridges the gap between the pre-training and fine-tuning for better transfer of knowledge. Experiments on three downstream tasks demonstrate that FactPEGASUS substantially improves factuality evaluated by multiple automatic metrics and humans. Our thorough analysis suggests that FactPEGASUS is more factual than using the original pre-training objective in zero-shot and few-shot settings, retains factual behavior more robustly than strong baselines, and does not rely entirely on becoming more extractive to improve factuality.
[{"first":"David","last":"Wan","middle":[]},{"first":"Mohit","last":"Bansal","middle":[]}]
Improving the Faithfulness of Abstractive Summarization via Entity Coverage Control
Abstractive summarization systems leveraging pre-training language models have achieved superior results on benchmark datasets. However, such models have been shown to be more prone to hallucinate facts that are unfaithful to the input context. In this paper, we propose a method to remedy entity-level extrinsic hallucinations with Entity Coverage Control (ECC). We first compute entity coverage precision and prepend the corresponding control code for each training example, which implicitly guides the model to recognize faithfulness contents in the training phase. We further extend our method via intermediate fine-tuning on large but noisy data extracted from Wikipedia to unlock zero-shot summarization. We show that the proposed method leads to more faithful and salient abstractive summarization in supervised fine-tuning and zero-shot settings according to our experimental results on three benchmark datasets XSum, Pubmed, and SAMSum of very different domains and styles.
[{"first":"Haopeng","last":"Zhang","middle":[]},{"first":"Semih","last":"Yavuz","middle":[]},{"first":"Wojciech","last":"Kryscinski","middle":[]},{"first":"Kazuma","last":"Hashimoto","middle":[]},{"first":"Yingbo","last":"Zhou","middle":[]}]
Enhancing Factual Consistency of Abstractive Summarization
Automatic abstractive summaries are found to often distort or fabricate facts in the article. This inconsistency between summary and original text has seriously impacted its applicability. We propose a fact-aware summarization model FASum to extract and integrate factual relations into the summary generation process via graph attention. We then design a factual corrector model FC to automatically correct factual errors from summaries generated by existing systems. Empirical results show that the fact-aware summarization can produce abstractive summaries with higher factual consistency compared with existing systems, and the correction model improves the factual consistency of given summaries via modifying only a few keywords.
[{"first":"Chenguang","last":"Zhu","middle":[]},{"first":"William","last":"Hinthorn","middle":[]},{"first":"Ruochen","last":"Xu","middle":[]},{"first":"Qingkai","last":"Zeng","middle":[]},{"first":"Michael","last":"Zeng","middle":[]},{"first":"Xuedong","last":"Huang","middle":[]},{"first":"Meng","last":"Jiang","middle":[]}]
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
We study generating abstractive summaries that are faithful and factually consistent with the given articles. A novel contrastive learning formulation is presented, which leverages both reference summaries, as positive training data, and automatically generated erroneous summaries, as negative training data, to train summarization systems that are better at distinguishing between them. We further design four types of strategies for creating negative samples, to resemble errors made commonly by two state-of-the-art models, BART and PEGASUS, found in our new human annotations of summary errors. Experiments on XSum and CNN/Daily Mail show that our contrastive learning framework is robust across datasets and models. It consistently produces more factual summaries than strong comparisons with post error correction, entailment-based reranking, and unlikelihood training, according to QA-based factuality evaluation. Human judges echo the observation and find that our model summaries correct more errors.
[{"first":"Shuyang","last":"Cao","middle":[]},{"first":"Lu","last":"Wang","middle":[]}]
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
State-of-the-art abstractive summarization systems often generate hallucinations; i.e., content that is not directly inferable from the source text. Despite being assumed to be incorrect, we find that much hallucinated content is actually consistent with world knowledge, which we call factual hallucinations. Including these factual hallucinations in a summary can be beneficial because they provide useful background information. In this work, we propose a novel detection approach that separates factual from non-factual hallucinations of entities. Our method is based on an entity’s prior and posterior probabilities according to pre-trained and finetuned masked language models, respectively. Empirical results suggest that our method vastly outperforms two baselines in both accuracy and F1 scores and has a strong correlation with human judgments on factuality classification tasks.Furthermore, we use our method as a reward signal to train a summarization system using an off-line reinforcement learning (RL) algorithm that can significantly improve the factuality of generated summaries while maintaining the level of abstractiveness.
[{"first":"Meng","last":"Cao","middle":[]},{"first":"Yue","last":"Dong","middle":[]},{"first":"Jackie","last":"Cheung","middle":[]}]
Improving Faithfulness in Abstractive Summarization with Contrast Candidate Generation and Selection
Despite significant progress in neural abstractive summarization, recent studies have shown that the current models are prone to generating summaries that are unfaithful to the original context. To address the issue, we study contrast candidate generation and selection as a model-agnostic post-processing technique to correct the extrinsic hallucinations (i.e. information not present in the source text) in unfaithful summaries. We learn a discriminative correction model by generating alternative candidate summaries where named entities and quantities in the generated summary are replaced with ones with compatible semantic types from the source document. This model is then used to select the best candidate as the final output summary. Our experiments and analysis across a number of neural summarization systems show that our proposed method is effective in identifying and correcting extrinsic hallucinations. We analyze the typical hallucination phenomenon by different types of neural summarization systems, in hope to provide insights for future work on the direction.
[{"first":"Sihao","last":"Chen","middle":[]},{"first":"Fan","last":"Zhang","middle":[]},{"first":"Kazoo","last":"Sone","middle":[]},{"first":"Dan","last":"Roth","middle":[]}]
Improving Factual Consistency in Summarization with Compression-Based Post-Editing
State-of-the-art summarization models still struggle to be factually consistent with the input text. A model-agnostic way to address this problem is post-editing the generated summaries. However, existing approaches typically fail to remove entity errors if a suitable input entity replacement is not available or may insert erroneous content. In our work, we focus on removing extrinsic entity errors, or entities not in the source, to improve consistency while retaining the summary’s essential information and form. We propose to use sentence-compression data to train the post-editing model to take a summary with extrinsic entity errors marked with special tokens and output a compressed, well-formed summary with those errors removed. We show that this model improves factual consistency while maintaining ROUGE, improving entity precision by up to 30% on XSum, and that this model can be applied on top of another post-editor, improving entity precision by up to a total of 38%. We perform an extensive comparison of post-editing approaches that demonstrate trade-offs between factual consistency, informativeness, and grammaticality, and we analyze settings where post-editors show the largest improvements.
[{"first":"Alex","last":"Fabbri","middle":[]},{"first":"Prafulla Kumar","last":"Choubey","middle":[]},{"first":"Jesse","last":"Vig","middle":[]},{"first":"Chien-Sheng","last":"Wu","middle":[]},{"first":"Caiming","last":"Xiong","middle":[]}]
Focus Attention: Promoting Faithfulness and Diversity in Summarization
Professional summaries are written with document-level information, such as the theme of the document, in mind. This is in contrast with most seq2seq decoders which simultaneously learn to focus on salient content, while deciding what to generate, at each decoding step. With the motivation to narrow this gap, we introduce Focus Attention Mechanism, a simple yet effective method to encourage decoders to proactively generate tokens that are similar or topical to the input document. Further, we propose a Focus Sampling method to enable generation of diverse summaries, an area currently understudied in summarization. When evaluated on the BBC extreme summarization task, two state-of-the-art models augmented with Focus Attention generate summaries that are closer to the target and more faithful to their input documents, outperforming their vanilla counterparts on ROUGE and multiple faithfulness measures. We also empirically demonstrate that Focus Sampling is more effective in generating diverse and faithful summaries than top-k or nucleus sampling-based decoding methods.
[{"first":"Rahul","last":"Aralikatte","middle":[]},{"first":"Shashi","last":"Narayan","middle":[]},{"first":"Joshua","last":"Maynez","middle":[]},{"first":"Sascha","last":"Rothe","middle":[]},{"first":"Ryan","last":"McDonald","middle":[]}]
Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling
While modern summarization models generate highly fluent summaries that appear realistic Zhang et al., 2020), these models are prone to generating non-factual and sometimes entirely fabricated content (Cao et al., 2018;Goodrich et al., 2019;Maynez et al., 2020). With the increasing adoption of language generation tools in user-facing products, such unreliability poses severe risks, including the spread of misinformation, panic and other potentially harmful effects (Ranade et al., 2021;Hutson et al., 2021).
Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling
Since it is difficult to control for factuality at training or inference time (Huang et al., 2021; The first vaccine for Ebola was approved by the FDA in 2019 in the US, five years after the initial outbreak in 2014. To produce the vaccine, scientists had to sequence the DNA of Ebola, then identify possible vaccines, and finally show successful clinical trials. Scientists say a vaccine for COVID-19 is unlikely to be ready this year, although clinical trials have already started.
Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling
Scientists believe a vaccine for Covid-19 might not be ready this year. The first vaccine for Ebola took 5 years to be approved by the FDA.
Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling
Scientists believe a vaccine for Ebola might not be ready this year. The first vaccine for Ebola took 5 years to be produced by the CBP. Figure 1: Model generated summaries often produce content which is factually inconsistent w.r.t. to the source. FACTEDIT rewrites these summaries by maintaining the abstractiveness but correcting factual errors. Dreyer et al., 2021), a popular approach to fix the factual inconsistencies is via post-editing generated summaries (Cao et al., 2020;. This allows summarization models to focus on fluency and content-relevance while improving factual consistency. However, there is no suitable data for training post-editing models to directly "translate" an incorrect summary to a correct one. Prior work constructed synthetic training data by introducing simple heuristic errors like replacing entities or numbers in reference summaries (Cao et al., 2020), but it is not clear whether such synthetic errors have sufficient coverage and accurately represent the types and distribution of actual errors made by language models. Further, with increasing language generation capabilities, models make more complex factual errors involving discourse structures and paraphrasing which cannot be easily captured with heuristics (Pagnoni et al., 2021). The goal of our work is to develop post-editing models  Figure 2: Architecture framework for FACTEDIT. Using masked versions of existing reference summaries, we use an infilling language model to produce alternative candidates for the mask position. We construct factually incorrect summaries by replacing the mask with the lower ranked candidates. Finally, we train a sequence-to-sequence model for fact correction using the synthetically constructed data.
Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling
that generalize over a wider range of factual errors (example in Figure 1) in generated summaries from diverse summarization model types.
Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling
We propose FACTEDIT-a novel approach to post-editing text, to control for content factuality in generated summaries. Rather than manually defining a list of heuristic errors, it incorporates a new algorithm to generate adversarial (non-factual) examples using infilling language models (Donahue et al., 2020). We use lower ranked beam-search candidates from the language model as a source for potentially factually-incorrect summary facts, thereby producing a set of plausible, likely, and fluent, incorrect synthetic summaries for a particular correct reference summary. In this way, we leverage the capabilities of large language models to produce multiple candidates of alternative, erroneous summaries. These examples, along with factually correct references, are then used to train a sequence-to-sequence fact-correction model that aims at generating a factually consistent version of the candidate summary ( §2).
Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling
We evaluate FACTEDIT on two datasets -CNN/DailyMail (Hermann et al., 2015) and XSum (Narayan et al., 2018) and across nine summarization models with the FRANK benchmark (Pagnoni et al., 2021) for evaluating various categories of factual errors in generated summaries ( §3). The two summarization datasets represent varied distributions of factual errors in models trained on them and hence constitute a good test bed to evaluate the generalizability of our model. We show that FACTEDIT substantially improves factuality scores across two metrics -Ent-DAE (Goyal and Durrett, 2021) and FactCC (Kryscinski et al., 2020). On the Ent-DAE metric, FACTEDIT improves results by ∼11 points (CNN/DM) and ∼31 points (XSum), and on the FactCC metric we show improvements of ∼6 points (CNN/DM) and ∼24 (XSum) points on average across models ( §4). Further, our analysis shows that FACTEDIT effectively corrects diverse error categories without the need for special heuristics or annotations ( §5). An important application of FACTEDIT is to audit summarization systems and facilitate their reliability.
Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling
Assume a summarization model trained to process a document d and generate a coherent and fluent summary 2 g ′ which has been shown to often misrepresent facts from the document. FACTEDIT is a fact correction model M C which takes the generated summary g ′ and document d, identifies factual errors and generates a rewritten summary g by correcting them (as outlined in Figure 2).
Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling
We present an adversarial data generation approach which leverages the power of pre-trained language models to produce fluent and complex factually incorrect summaries. We train an infilling language model M I using documents from summarization training data and use the model to introduce diverse factual errors in sentences from them ( §2.1). Using the trained model, we introduce factual errors in reference summaries of the training data r producing an incorrect summary r ′ resulting in a synthetic dataset {r ′ , r, d} train of erroneous summaries mapped to their corrected versions (pink section in Figure 2). We train a sequence-to-sequence model M C for factual error correction using the generated synthetic data ( §2.2). Finally, we use the trained correction model to rewrite model generated summaries g ′ producing a corrected version g ( §2.3 -green section in Figure 2).
Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling
Our data generation process leverages infilling language models (Donahue et al., 2020) to produce candidates to fill masked phrases in a summary sentence. We mask parts of the input and use the infilling model to generate multiple candidates for the masked position. We then use lower order beam candidates as potential incorrect candidates to generate an incorrect version of the input. We hypothesize that, given the relevant context of a source document, a strong language model generates relevant and factual sequences at higher probabilities, compared to lower probability sequences. For the infilling model, we hypothesize that the lower ranked candidates are often alternative phrases of similar types (in case of entities) or parts-of-speech which are plausible but often not factually correct. Motivated by prior work (Goyal and Durrett, 2020) using lower ranked beam search candidates as a source for adversarial data, we use the lower ranked candidates to construct erroneous summaries from reference summaries.
Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling
Training: Our infilling model M I is trained to take a masked sentence s masked and its relevant context ctx as input and generate a correct phrase to fill in the masked span. To train M I , we construct a dataset using documents d from the training data of existing summarization datasets. For each sentence s in the first-k (k=5) positional sentences of a document d, we identify the subjects, objects and relations {sub, obj, rel} in them using OpenIE (Banko et al., 2007). By iteratively masking each phrase p in {sub,obj,rel}, we create a masked query s masked and its corresponding context ctx by removing the masked sentence from the document, resulting in our training data {s masked , p, ctx}, where p is the masked span text. We train a sequence-to-sequence model M I on this data which takes s masked [SEP] ctx as input and learns to generate p as the output. We intentionally use only sentences from the document as masked queries and do not use sentences from the reference summaries, to ensure that the model does not memorize phrases from the references. Thus, when applied to unseen reference sentences during inference, the model will produces richer beam search candidates.
Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling
Adversarial Data Generation: We use the trained infilling model to generate the synthetic dataset for fact correction using the document reference pairs {d, r} train from the summarization training data. For each sentence in the reference s r , we use OpenIE to extract {sub, obj, rel} and iteratively mask one phrase at a time to construct masked sentences s masked from the references. We provide this masked reference summary sentence and document d as input to the model and perform beam-search decoding for generation. We then consider lower ranked beam candidates (rank= [5,15]) 3 as non-factual alternatives for the corresponding masked phrase. We then use these candidates as the replacements for the mask producing an erroneous summary r ′ . Running this on the {d, r} train training data, we construct a synthetic data {r ′ , r, d} train of factually incorrect summaries paired with their correct version where r ′ and r differ by an incorrect phrase. To train the model to not perform any corrections on factual summaries, we keep original reference summaries for 20% of the data points (r ′ = r).
Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling
Using the parallel data {r ′ , r, d} train produced by the above infilling method, we train models for factual error correction. In contrast to prior work which used pointer based models to copy phrases from the source document, we use a sequence-tosequence model like BART  to preserve the abstractive content in the input. The model M C is trained with an erroneous reference summary sentence s r ′ produced by the infilling data generator and the corresponding document d as input and the correct reference summary sentence s r as output. A straightforward option is to provide s r ′ , d concatenated as inputs to the model. But we hypothesize that providing the right context can help the model better correct the errors. Below we outline input structures that provide better context in the input: Relevant Supporting Passages: To help the model better connect the relevant facts in the source document to the summary sentence being corrected, we experiment with providing only the most relevant parts of the document as input context instead of the entire document. Using a scoring function (ROUGE), we identify sentences from the document which have high overlap with the generated summary sentence and extract the top-k (k=3 for our work) such sentences. We provide these sentences along with a window of w k (w k =2) sentences before and after each as the input context to the model. Surrounding Summary Context: While simple errors like incorrect entities can be detected and corrected with only the context of the current sentence being corrected, more complex discourse level errors like incorrect pronouns require the context of the rest of the sentences of the summary. To enable this, we additionally give the complete generated summary (other sentences from the summary) as additional context. For single sentence summaries like headline generation, this does not change the original setting, but for longer summaries this setting helps with discourse level errors. In essence, our model M C takes the input as Incorrect Reference Sentence (s r ′ ) [SEP] Full Reference Summary (r ′ ) [SEP] Relevant Passages and generates the corrected summary r as output.
Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling
Our trained fact correction model M C can be directly applied to any model-generated summaries g ′ , without access to the underlying model. For each sentence in a generated summary, we identify the relevant passages using ROUGE and provide it as an input to the model (in the form Gen-
Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling
We use two news summarization datasets CNN-DailyMail (Hermann et al., 2015) and XSum (Narayan et al., 2018). The two datasets have been extensively studied for the factual consistency in their generated summaries across a variety of models (Goodrich et al., 2019;Cao et al., 2018). Reference summaries from CNN/DM are longer, having on average three sentences, and more extractive in nature. XSum on the other hand has shorter, single sentence summaries and is significantly more abstractive in nature. The summaries in these datasets are qualitatively different, and hence models trained on the two datasets present varied levels of challenges in maintaining factual consistencies.
Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling
Prior work have studied summaries generated from different language models and characterized the factual errors in them (Pagnoni et al., 2021). Generated summaries on the CNN/DM dataset are more extractive in nature and hence are more factual (∼70% of summaries are factual) than the more abstractive generated summaries of XSum (∼20% of summaries are factual). The longer summaries in CNN/DM display discourse level inconsistencies while summaries from XSum often hallucinate content which is not supported by the source document. Hence, the two datasets present a varied setting for evaluating the efficacy of our model across different kinds of errors. For our main evaluation, we evaluate the overall capability of FACTE-DIT in correcting errors in summaries generated by a BART model.
Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling
We further evaluate our model on the FRANK benchmark (Pagnoni et al., 2021) which contains generated summaries obtained using multiple language models for both datasets annotated with human judgements on their factuality and the category of factual error. As different language models have different distribution of factual error types, this evaluation helps us study the generalizability of FACTEDIT in correcting errors across them. 4 For the CNN/DM dataset, it contains model outputs from a LSTM Seq-to-Seq model (S2S) (Rush et al., 2015), a Pointer-Generator Network (PGN) model (See et al., 2017), a Bottom-Up Summarization (BUS) model (Gehrmann et al., 2018), a Bert based Extractive-Abstractive model (BertSum) (Liu and Lapata, 2019) and a jointly pretrained transformer based encoder-decoder model BART . For the XSum dataset, it contains model outputs from a Topic-Aware CNN Model (Narayan et al., 2018), a Pointer-Generator Network (PGN) model, a randomly initialized (TransS2S) (Vaswani et al., 2017) and one initialized with Bert-Base (BertS2S) (Devlin et al., 2019).
Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling
Setup: We use OpenIE (Banko et al., 2007) to pre-process each summary and extract subject, object, predicate triples for each summary sentence. We use BART-base  as our sequence-to-sequence model for the infilling based data generator and the fact correction model. Both models were trained with a batch size of 48, a learning rate of 3e-5, and warm-up of 1000 for 1 epoch. The maximum input sequence length was 512 and maximum output sequence length was 128. Using the infilling data generator, we generate 1233329 negative, 308332 positive examples for CNN/DM and 724304 negative, 181076 positive, examples for XSum as training data for fact correction. Models were trained on 4 Nvidia GeForce GTX TITAN X GPUs and each training run took ∼15 hours. All hyperparameters were chosen based on generated dev set ROUGE-L (Lin, 2004) on each dataset. Evaluation Setup: Evaluating factual consistency of generated summaries is challenging, with relatively recent metrics developed to detect it. These metrics unfortunately do not correlate highly with human judgements yet. We therefore evaluate our model using two metrics -FactCC (Kryscinski et al., 2020) and Ent-DAE (Goyal and Durrett, 2021); each captures different error types. FactCC is a binary classifier, trained on a synthetic, heuristic error dataset, which is better at detecting simple semantic errors like incorrect entities or numbers. Ent-DAE is a classifier trained on synthetic data constructed using the dependency structure of the text. In addition to semantic errors, it is better at detecting more complex discourse-level errors (Pagnoni et al., 2021). We also report ROUGE (Lin, 2004) to evaluate if our model maintains the fluency of summaries. While ROUGE is less correlated with factuality (Pagnoni et al., 2021;Maynez et al., 2020), it helps evaluate if the corrected summary is fluent and aligned with the reference summary. However, with factual corrections of outputs we expect small drops in ROUGE, since generation models were specifically optimized to maximize ROUGE presumably at the expense of factuality.
Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling
Our evaluation has two settings: i) FACTEDIT -correct all generated summaries in the test set and ii) FACTEDIT + FactCC Filter (FF) -using the FactCC metric we identify factually incorrect summaries, and only correct the incorrect ones. Baselines: We compare our approach with (Cao et al., 2020) as the baseline. The baseline uses a heuristic set of rules proposed by Kryscinski et al. (2020) to introduce simple errors (Entity, Number, Date, and Pronoun) in reference summaries and trains a BART-base model for error correction. Comparing our model with (Cao et al., 2020) helps us evaluate the benefit of our Infilling LM based adversarial data generator. 5
Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling
We first evaluate FACTEDIT's ability to correct errors in summaries generated by a BART-base summarization model on the entire test set. We first generate summaries using a BART-base model finetuned on each dataset and then provide the generated summaries and their corresponding source documents as inputs to FACTEDIT for correction. Table 1 shows results for this experiment. Our results show that correcting factual errors using our model improves the factuality results. The baseline model performs poorly with the FactCC metric showing lower scores than the BART model generated summaries, especially in the more abstractive XSum setting. The DAE metric for the baseline model is slightly lower than the BART model scores in the CNN/DM setting and has no improvement in the XSum setting showing that it does not perform corrections on complex errors. These results confirm our hypothesis that the baseline model trained on adversarial data based on heuristic errors does not transfer well to real errors in model generated summaries. In contrast, our model improves both metrics across both datasets. On the more challenging XSum dataset, our model has a ∼17 point improvement on FactCC and ∼0.  improvement on DAE over the baseline model. The BART generated summaries on CNN/DM are ∼70% factual and hence using the FactCC Filter to correct only non-factual summaries helps improve results on FactCC. As XSum has more than 80% non-factual summaries, the FactCC filter does not change results and correcting all generated summaries is beneficial. In Table 6 we present examples of corrections made by FACTEDIT and present a discussion in §A.
Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling
Prior works have shown that improving factual consistency in summaries leads to a drop in ROUGE scores (Maynez et al., 2020;Cao and Wang, 2021;Cao et al., 2020). Our ROUGE results do not drop significantly and are consistent with prior work. These results show that our model does not significantly change the summaries and the corrected summaries contain the relevant information w.r.t. to the source. Table 2 shows results of using FACTEDIT to correct summaries generated by different types of language models using the FRANK benchmark (Pagnoni et al., 2021). We provide the generated summaries collected in the benchmark along with their source document as input to our trained fact corrector. This setting evaluates the generalizability of our adversarial training data in handling different error distributions from different summarization models.
Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling
Our results show our model significantly improves the factuality in generated summaries across 8 out of 9 test models. The FactCC Filter helps improves results in CNN/DM setting but does not change results in XSum similar to results in §4.1. In the more extractive CNN/DM setting, fact correction improves FactCC scores by ∼5.3 points and DAE scores by ∼10.9 points on average across models. In the more challenging and abstractive XSum dataset, we improve FactCC scores by ∼24 points and DAE scores by ∼31 points on average. Our results show that our model trained using Infilling LM based adversarial data is able to generalize and correct errors in generated summaries across different model types. Further, the significant improvement in XSum suggests that using LMs to generate factually incorrect candidates produces rich negative examples which help correct errors in Figure 3: Performance of FactEdit across different error categories in comparison to baseline (Cao et al., 2020) . FACTEDIT improves the percentage of factual summaries across diverse types of factual errors. more abstractive summaries.
Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling
Pretrained models like BART, BertSum and BertS2S have improved generation capabilities and make lesser mistakes in generating the right entity or predicate and more mistakes in discourse structuring (Pagnoni et al., 2021). FACTEDIT correspondingly shows larger improvements in DAE scores than FactCC scores in these pretrained models. The Pointer-Generator model being highly extractive in nature scores highly in factuality metrics in the CNN/DM setting and FACTEDIT reduces results in this setting showing that our model is not beneficial in copy-based model settings. On the other hand, in the XSum setting, the base Pointer-Generator model scores poorly and correcting factual errors in them improves factuality scores. Non-pretrained sequence-to-sequence models like Seq2Seq and TransSeq2Seq score poorly in both ROUGE and Factuality scores due to their limited language generation capabilities. By correcting factual errors in them, we improve factuality metrics significantly without changes in ROUGE, indicating that the gains are due to fact correction and not just rewriting the summary using a strong language model.
Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling
The FRANK benchmark proposes a typology of three coarse categories of error types and collects human annotations on the error category: i) Semantic Frame Errors -This category covers factual errors in a sentence due to incorrect entity or predicate being generated ii) Discourse Errors -This covers discourse level factual errors like incorrect pronouns or sentence ordering iii) Content Verifiability Errors -This category is for errors whose factuality cannot be judged either due to grammatical errors or hallucinated content. We evaluate our model on its ability to correct different types of errors. We use the generated summaries from the best pretrained model in FRANK for each dataset -BART for CNN/DM and BertS2S for XSum. For each subset of summaries of a particular error type, we correct the summaries using FACTEDIT and report the percentage of factual summaries in the output as predicted by Ent-DAE. We compare FACTE-DIT with the baseline to study whether our model improves error correction for each type.
Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling
From Figure 3, we see that across both datasets FACTEDIT increases the percentage of factual summaries across all three error categories, showing that the data generation process in FACTEDIT can generalize across multiple error types without the need for special heuristics or annotations. We see the largest improvements in the Semantic Frame Error category with an increase of ∼ 8 points on CNN/DM and ∼ 13 points on XSum. On the more complex Discourse Errors we see an improvement of ∼ 5 points on both datasets. Finally, on Content Verifiability Errors, we see a ∼ 8 point improvement on CNN/DM and ∼ 2 point improvement on XSum. XSum has a high proportion of hallucination errors and our results highlight the challenge in correcting this error type.
Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling
It is not always feasible to train specialized fact correction models for each dataset or style of summaries. While CNN/DM and XSum contain documents of the news domain, they both have different summary characteristics. Certain applications might benefit from a single model which can generalize to different summary styles. We evaluate   the ability of FACTEDIT trained on CNN/DM data (FACTEDIT FF -CNN Model) to transfer and correct summaries generated for XSum documents using FRANK benchmark. Table 3 shows results for this experiment. Our results show significant improvement in factuality scores across all model types in this setting, showing that our data generation process produces rich and diverse factually incorrect examples which can generalize to factual errors in other data settings. By using only the source documents, our training data is agnostic of the styles, lengths and characteristics of reference summaries and hence is able to generalize to the headline style abstractive summaries of XSum.
Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling
To further study whether the factuality corrections performed by our model align with human expectations of automated summaries, we conduct a human study. Two annotators evaluated 20 randomly sampled summaries generated from the test set of the XSum dataset using the BertS2S model and corrected by FACTEDIT and the baseline. The annotators were shown the entire source document and one corrected summary at a time and asked to rate the fluency and factuality of the summary on a 1-5 Likert scale. In manual evaluation, annotators rated FACTEDIT an average of 3.3 on factuality and 4.8 on fluency, compared to the baseline which was rated 3.1 and 4.6 scores respectively, showing that FACTEDIT improves on both factuality and fluency.  
Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling
Our model corrects each sentence in a summary given context of the rest of the summary and relevant passages in the source document. We ablate this setup by removing parts of the context one at a time. In Table 5 we present the results. We observe a a drop in results when using the entire summary as context (-RelevPass) and when removing the context of the summary in which the sentence occurs (-SummCtxt). Our results show the importance of having the appropriate context to enable the model to perform fact correction well.
Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling
We present an adversarial data generation process to generate rich synthetic data for a post editing model, which can be applied to correct factual errors generated summaries. Our data generation process leverages Infilling Language Models to produce alternative candidate summaries. Using the generated data, we train models to rewrite summaries by correcting factual errors in them.
Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling
Through extensive experiments across two datasets and nine models, we show that our fact corrector model improves the factual consistency of the summaries, making them more reliable.
Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling
Our model is trained to rewrite generated summaries by correcting factual errors in them. A limitation in our current setup is accurate detection of factual errors. We rely on off-the-shelf metrics for identifying summaries with factual errors to correct. Our model does not perform detection and correction together and often rewrites correct summaries as well if fed to the model. Therefore for settings like CNN/DM, it's beneficial to filter summaries using a factuality metric before giving summaries to our model as input. As our fact corrector is a sequence-to-sequence model, it could potentially introduce new factual errors in the summaries. It is essential to use factually detectors to ensure summaries are factual before real world usage of any corrected summary.
Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling
State-of-the-art language generation models, including summarization, are not yet powerful enough to facilitate fine-grained control over generated content. This leads to problems with content fidelity and safety; our work aims to ameliorate issues related to factual reliability of the models. However, existing approaches, including ours, cannot guarantee this yet. Furthermore, there is a risk of dual use, since the same techniques can be used to post-edit models to produce non-factual, harmful content to mislead, impersonate, or manipulate opinions. Future research should focus on developing better defenses methods against mis-using language generators maliciously.
Factual Error Correction for Abstractive Summarization Models
Self-supervised methods have achieved success in a wide range of NLP tasks, and automatic summarization is no exception (Liu and Lapata, 2019;Lewis et al., 2019;Zhang et al., 2019a;Shi et al., 2019;Fabbri et al., 2019). These state-of-theart abstractive summarization models typically finetune pre-trained transformer-based models on a summarization dataset (Vaswani et al., 2017). Despite significant improvements over previous methods in terms of automatic evaluation scores such as ROUGE (Lin, 2004), ensuring factual consistency of the generated summary with respect to the source remains challenging. For example, 1 Our data and code is available at https://github.com/mcao610/Factual-Error-Correction Source: Jerusalem (CNN)The flame of remembrance burns in Jerusalem, and a song of memory haunts Valerie Braham as it never has before. This year, Israel's Memorial Day commemoration is for bereaved family members such as Braham. "Now I truly understand everyone who has lost a loved one," Braham said. (...) Original: France's memorial day commemoration is for bereaved family members as braham. (inconsistent) After Correction: Israel's memorial day commemoration is for bereaved family members as braham. (consistent) Table 1: An example of an inconsistent systemgenerated summary and the output summary from our correction model. In this case, "France" is successfully corrected as "Israel". Cao et al. (2018) claims that about 30% of summaries generated by abstractive models contain factual errors, which greatly limits their practicality.
Factual Error Correction for Abstractive Summarization Models
Different approaches have been proposed to detect or ensure the factual consistency of generated summaries, including using fact extraction or applying attention on fact triples (Cao et al., 2018;Zhang et al., 2019b;Goodrich et al., 2019), applying natural language inference or question answering models for consistency checking (Falke et al., 2019;Li et al., 2018;Wang et al., 2020) and training the model on artificial datasets (Kryściński et al., 2019). Most of these approaches either require a high-quality fact extraction model or they only focus on factual consistency evaluation. Improving factuality correction by editing inconsistent parts in generated summaries is a direction that has not been explored much.
Factual Error Correction for Abstractive Summarization Models
In this work, we propose a model to improve the factual consistency of system summaries with post-editing correction (Table 1). Our model takes a draft summary that is generated by an abstractive summarization model and produces a corrected final summary, conditioned on the source document. In addition, our trained corrector can be used as an evaluation model for factual consistency of abstractive summaries, with the assumption that a generated summary is inconsistent if our corrector decides to make edits. To teach the model to correct errors, we train it with artificial data that has factual errors introduced using heuristics proposed by Kryściński et al. (2019).
Factual Error Correction for Abstractive Summarization Models
The empirical results based on automatic and human evaluations indicate that our model not only corrects factual errors in summaries, it is also a better and more reliable factuality evaluation model than FactCC (Kryściński et al., 2019), a recent proposed factuality evaluation method. In a downstream setting where we apply the corrector to the output of an abstractive summarizer, we find that our corrector is able to accurately correct errors in the generated summaries. However, the overall recall on correcting factual errors in real system summaries remains low, suggesting the errors introduced by heuristics have a different distribution than errors made by abstractive summarization systems.
Factual Error Correction for Abstractive Summarization Models
Previous work on factual consistency in abstractive summarization can be divided into two categories: abstractive summarization models tailored towards factual consistency (Cao et al., 2018;Zhang et al., 2019b;Li et al., 2018), and evaluation models for factual consistency in abstractive summarization (Goodrich et al., 2019;Falke et al., 2019;Kryściński et al., 2019;Wang et al., 2020). Cao et al. (2018) proposed a dual attention module in an abstractive summarizer that attends to both the source document and to relation triples extracted from the document. Zhang et al. (2019b) propose to improve their abstractive summarization model by optimizing fact scores defined in radiology reports with reinforcement learning methods. Li et al. (2018) jointly train their model's encoder on summarization and NLI tasks. Guo et al. (2018) train an abstractive summarization system with the auxiliary tasks of question and entailment generation and show that their generated summaries are less likely to produce extraneous facts. Kumar and Cheung (2019) show that neural abstractive summarizers often assign higher posterior likelihood to perturbed contrastive summaries that are inconsistent with the source text than to human-written gold-standard ones. Concurrently to our work, Zhu et al. (2020) recently proposed a fact-aware summarization model that uses a knowledge graph. They use a pre-trained corrector module to modify generated summaries. Concurrent to our work, Dong et al. (2020) proposes factual correction models that leverages knowledge learned from question answering models via span selection. Their models employ single or multimasking strategies to either iteratively or autoregressively replace entities.
Factual Error Correction for Abstractive Summarization Models
In terms of evaluating abstractive summarization models for factual consistency, Goodrich et al. (2019) proposed a metric to check factual consistency by checking the overlapped fact triples between a source document and generated text on Wikidata. Falke et al. (2019) shows that factual error detection is a difficult task on its own and adapting entailment models for factual error detection do not offer the desired performance. Kryściński et al. (2019) finetune a BERT model on heuristically-created data with six types of rule-based text transformations for factual consistency checking. Wang et al. (2020) propose a framework for measuring inconsistencies in abstractive summarization by answering questions based on both generated summaries and documents.
Factual Error Correction for Abstractive Summarization Models
In this section, we describe our procedure of introducing artificial errors in the datasets for training and propose our end-to-end error corrector model.
Factual Error Correction for Abstractive Summarization Models
Inspired by a recent study of error types made by state-of-the-art summarization system, we artificially created a weakly-supervised training dataset based on the text transformations proposed by Kryściński et al. (2019).
Factual Error Correction for Abstractive Summarization Models
Given a source text d and the reference summary s, we corrupt the reference summary into an inconsistent summary s ′ with a randomly sampled corruption rule (described below) with probability α; otherwise, we keep s ′ = s with probability 1 − α. We set α = 0.3 to match the factuality error rate in real abstract summaries based on a recent study (Cao et al., 2018). The training data consists of triplets (s ′ , s, d).
Factual Error Correction for Abstractive Summarization Models
Error Corruptions Four types of errors are used to create the inconsistent summaries: En-   (2019), we corrupt the reference summary rather than sentences sampled from the source document.
Factual Error Correction for Abstractive Summarization Models
In the first four types of error constructions, we utilize a swapping strategy to introduce errors. For Entity, Number, and Date swapping, one entity in the reference summary is selected and swapped with another random entity of the same type 2 in the source document. For Pronoun swapping, one pronoun was extracted and swapped with another one of a matching syntactic case. Table 2 shows one example of a corruption.
Factual Error Correction for Abstractive Summarization Models
With the artificial training data consisting of triplets (s ′ , s, d), the goal of the corrector is to generate the correct summary s based on the inconsistent summary s ′ and the source d. This can be expressed as a problem of maximizing the likelihood of P (s|s ′ , d) in an encoder-decoder model. We concatenate s ′ and d as input to the encoder (s ′ and d are separated by a separation token) and train the decoder to generate s.
Factual Error Correction for Abstractive Summarization Models
We use BART (Lewis et al., 2019) as the basis of our summary corrector because of its demonstrated level of performance on conditional text generation tasks. BART is a sequence-to-sequence
Factual Error Correction for Abstractive Summarization Models
Acc.  auto-regressive transformer model that is pretrained as a denoising auto-encoder. One appealing aspect about BART is that it is pre-trained on a denoising task. Specifically, given an input sentence that is corrupted by text infilling, token deletion as well as other text transformations, BART is trained to output the original sentence. This pretraining task is similar to our summary correction task in which we can regard the corrupted or generated summary as the noisy input and in this case the noise is the inconsistent content in the summary.
Factual Error Correction for Abstractive Summarization Models
We evaluate our model on two tasks: factual consistency checking and error correction.
Factual Error Correction for Abstractive Summarization Models
Factual consistency checking For this task, the model needs to classify each original input summary as consistent or inconsistent with respect to the source text. It is thus a binary classification task for which we report accuracy, as well as precision, recall, and F1. We interpret the output of our corrector model as a classification decision as follows. If the corrector makes any change to the original input summary, we consider this to be a prediction of the inconsistent class. Otherwise, the corrector makes no change and we consider this a prediction of the consistent class.
Factual Error Correction for Abstractive Summarization Models
Error correction For this task, the model must correct inconsistencies in the original summary (in any) with respect to the source text.
Factual Error Correction for Abstractive Summarization Models
We define correction accuracy as the proportion of original summaries that are correctly changed by our corrector. On our artificial test set, an input summary is considered successfully corrected if the corrected summary matches the ref-erence summary exactly. For the K2019 dataset, no reference corrections are available. We instead conducted a human evaluation to check the consistency of the corrected output. We read the original and corrected summaries as well as the source document to determine whether a summary is successfully corrected by our model.
Factual Error Correction for Abstractive Summarization Models
We use two datasets for our experiments. The first is the dataset of artificial corruptions described in Section 3.1, which we create by taking samples from the CNN/DailyMail dataset. There are in total 287,227 samples in the training set, and we corrupted 30% of them (85,583). This results in 16,858/35,113/13,408/20,204 date/entity/number/pronoun corrupted samples respectively. We refer the other 201,644 training samples as clean samples. We also create artificial validation and test set for model selection and evaluation. In the test set, there are 5,780 corrupted samples and 5,710 clean samples.
Factual Error Correction for Abstractive Summarization Models
The second dataset we use is the K2019 test set of Kryściński et al. (2019). This dataset contains 503 summaries generated by different recent neural abstractive summarizers, which have been manually labeled for whether they contain an inconsistency.
Factual Error Correction for Abstractive Summarization Models
We evaluate our model on both datasets. We did not use baselines for the artificial test set since it is simply used as a check to demonstrate our model's performance in the artificial setting. The more meaningful evaluations are on K2019 consistency checking and error correction.
Factual Error Correction for Abstractive Summarization Models
We use the BART implementation from fairseq as the basis of our corrector. 3 The pre-trained BART model is fine-tuned on our training dataset for 10 epochs as described in Section 3.2. The learning rate is set to 3e-5. All our experiments is done on 4 NVIDIA Tesla V100 GPUs. The training process takes about 12 hours.  model is able to identify these artificially injected errors.
Factual Error Correction for Abstractive Summarization Models
For error correction, among the 5780 corrupted summaries in the test set, 62.13% are corrected by the model to exactly match the reference summary. For the 5710 clean summaries, the model made changes to 26.27% of them, which results in 73.73% correction accuracy on clean summaries. These results show that the model is able to correct majority of the test samples even under our strict evaluation measure.
Factual Error Correction for Abstractive Summarization Models
K2019 Table 4 shows the consistency checking results on the K2019 test set. Compared with FactCC, our model improves performance by more than 10% higher in accuracy and 0.11 in macro F1-score. This result is interesting considering that the model is trained on a generation task rather than on classification.
Factual Error Correction for Abstractive Summarization Models
As for correction performance, Table 6 shows the evaluation result of our human evaluation. Among 62 inconsistent summaries in the test set, the corrector model made changes to 19 summaries, of which 11 were successfully corrected and 7 remained inconsistent. For the remaining 441 consistent summaries in the test set, changes are made to 39 summaries and the model changed the meaning of 5 samples. In conclusion, with 17.74% probability that our model can successfully correct an inconsistent summary and 1.13% probability that it will corrupt a consistent one. Compared with the correction rate of 62.13% on the artificial test set, much lower correction rate on the real test set suggests that there is still a gap between the two settings. The error types in the training set are not able to represent the diverse errors made by summarization systems.  Output Analysis Table 5 shows several input and output summaries of our corrector model together with the source document fragments. In the second example, the model correctly replaced 147 with 19, but was not able to correctly remove "including 142 students", which is a larger modification to the original summary. More examples can be found in the Appendix.
Factual Error Correction for Abstractive Summarization Models
In this paper, we proposed a novel approach to correct inconsistent content in summaries generated by abstractive summarization models. We train an end-to-end correction model with artificial examples created by corrupting reference summaries. Our model achieved promising performance on our artificial test set and outperformed previous models on the manually annotated test set by wide margins. Our human evaluation indicates that our model is able to correct some factually inconsistent summaries generated by abstractive summarization model. However, low recall on the inconsistent summaries and false positive samples remain as challenges. 
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
Abstractive summarization aims at generating short summaries that capture the essentials of a long document. Research in this challenging task has made significant progress with the help of large pre-trained models (Lewis et al., 2020;Raffel et al., 2020;Zhang et al., 2020). However, current models suffer from the crucial problem of hallucinations (Maynez et al., 2020), where a summary contains facts or entities not present in the original document. Such unfaithful generation raises the question of whether the models can be trustworthy and used safely for real-world applications. To tackle this problem, many approaches propose postprocessing models (Chen et al., 2021;Dong et al., 2020;Liu and Liu, 2021), but such methods are often constrained by external resources to train additional correction or selection models. An alternative line of works focuses on learning factuality directly during fine-tuning by filtering nonfactual training data (Goyal and Durrett, 2021;Nan et al., 2021) or, most recently, incorporating contrastive learning (Cao and Wang, 2021) to encourage generating faithful summaries.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
In this work, we propose FACTPEGASUS, a model that addresses the problem of hallucinations for abstractive summarization holistically, by incorporating factuality into the whole training pipeline: We tackle the lack of factuality objective in pre-training and the presence of hallucinations in the downstream dataset during finetuning. Current pre-training objectives focus on improving the quality of the generated output in the downstream tasks but often overlook the factuality aspect. Thus, we explore incorporating factuality into the pre-training objective of PEGASUS (Zhang et al., 2020) (a state-of-the-art abstractive summarization model). The original objective, gap sentence generation (GSG), transforms any text into a pseudo-summarization dataset by selecting important sentences using ROUGE (Lin, 2004) as output summaries. We explore strategies for combining ROUGE and the factuality metric FactCC (Kryscinski et al., 2020) as the selection criteria, so that the model learns to generate sentences that cover the most important information of the input document as well as remain faithful to it.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
Next, we propose three complementary modules that further address factuality problems during fine-tuning: (1) Corrector that removes hallucinations existing in reference summaries, allowing training on the full training set without learning unfaithful behaviors; (2) Contrastor that encourages the model to better differentiate factual summaries from nonfactual ones by paying attention to the document using contrastive learning; (3) Connec-tor, a special mask-token fine-tuning technique enabled by the GSG-style objective, that simulates the pre-training task during fine-tuning by inserting the mask token into the input document so that the pre-trained model can adapt its knowledge of generating factual summaries directly to the downstream tasks. The connector, corrector, and contrastor address the input, output, and training objective of the downstream task, respectively, and the combination of the components reduces potential confounding problems that cannot be addressed by a single module. We show that the full model improves three factuality metrics, the token and sentence error of DEP Entail (Goyal and Durrett, 2021) and FactCC, on the downstream datasets of XSum (Narayan et al., 2018), WikiHow (Koupaee and Wang, 2018), and Gigaword (Rush et al., 2015). Most notably, FACTPEGASUS outperforms existing factualityaware summarization models by more than 40% and 34% on XSum for token error and FactCC, respectively. Ablation studies show the usefulness of each of our fine-tuning components as well as the additive gain of combining our complementary modules, and human evaluation confirms that FACTPEGASUS generates significantly more factual summaries over strong baselines.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
Finally, we perform a detailed analysis of FACT-PEGASUS, which points to several important observations regarding learning and maintaining factuality: (1) Zero-shot setting demonstrates the utility of our factuality-aware pre-training objective, as our model outperforms PEGASUS (which uses the original objective) on all three factuality metrics when evaluated directly on the downstream task without any supervised training data. Few-shot experiment indicates that even a small number of nonfactual examples can have a strong negative impact on factuality and can nullify much of the gain from factuality pre-training, highlighting the importance of ensuring factuality during fine-tuning. (2) Factuality dynamics (Goyal et al., 2022) further shows that FACTPEGASUS exhibits a lesser degree of factuality degradation than what is observed for BART-base. (3) Factuality vs abstractiveness tradeoff curve reveals that FACTPEGASUS effectively improves factuality by not simply relying on the increase in extractiveness.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
To summarize, our contributions are as follows:
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
1. We propose a factuality-aware pre-training objective for abstractive summarization and study the effect of different sentence selection strate-gies on downstream factuality.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
2. We introduce three complementary components for improving factuality during fine-tuning that correct hallucinations present in the training set, discourage unfaithful generation during training, and bridge the gap between pre-training and finetuning. The full model consistently achieves better factuality scores than strong baselines on three downstream abstractive summarization tasks, confirmed by human evaluation.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
3. We conduct thorough factuality analysis and show that FACTPEGASUS generates more factual summaries with no or little supervision, slows down factuality degradation observed for current models, and improves factuality not by becoming more extractive.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
We describe our training procedure consisting of pre-training with a factuality-aware objective (Section 3.1) and fine-tuning with three complementary modules for improving factuality (Section 3.2).
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
Recent exploration of good pre-training objectives for abstractive summarization aims at achieving high quality on downstream tasks, often in terms of ROUGE. However, few have analyzed the effect of pre-training objective on factuality. We focus on incorporating this aspect into the pre-training objective of PEGASUS, gap sentence generation (GSG), since PEGASUS achieves state-of-the-art performance on the downstream abstractive summarization tasks. The GSG objective transforms text documents into a pseudo-summarization dataset by selecting important sentences as the output summary, which are subsequently masked out in the original text. The best strategy determines the importance by calculating ROUGE-1 between each chosen sentence and the rest of the document. While the original strategy selects sentences that contain the most unigram overlap, there is no guarantee that the selected sentences are faithful to the rest of the document. We provide an illustrative example in Figure 1a, where the original objective selects sentence C due to its high ROUGE-1 score. However, this sentence is not a faithful summary to the rest of the document as the other sentences concern with the fire in Seattle while only sentence C talks about the fire in Denver.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
To address this problem, we extend this objective, which we call factual GSG (factGSG), where we additionally measure the importance of the sentences according to factuality. We use FactCC (Kryscinski et al., 2020) as the factuality criteria when selecting the summary sentences, as it correlates highly with human factuality judgment (Pagnoni et al., 2021) and is relatively fast to compute. FactCC produces a binary prediction where a score of 1 indicates that the selected sentence is consistent with the rest of the document. Another change in factGSG is the choice of gap sentence ratio, which determines the percentage of sentences in the text that will be selected as the summary. Instead of selecting 30% of the text document as output summary, we only select one sentence, as selecting more sentences will inevitably increase the possibility of hallucinations.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
Formally, given a document D of n sentences, D = {x 1 , x 2 , ..., x n }, we select the top-scoring sentence as the output summary, where the score of each sentence x i is calculated by:
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
Going back to the example in Figure 1a, FactCC assigns a score of 0 to the nonfactual sentence C because the fire in Denver is not entailed by the other sentences. This results in sentence A scoring higher than the nonfactual sentence, and thus overcomes the problem in the original objective.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
Although the typical approach of updating all the model's parameters during fine-tuning adapts well to the downstream task, the model suffers from imitative falsehood (Lin et al., 2021): The model learns to generate similar hallucinations present in the downstream dataset, and even completely forgets its factual behaviors learned during pretraining. This is especially problematic for datasets (b) Fine-tuning Figure 1: Illustration of FACTPEGASUS. For pre-training (a), we use the factGSG objective introduced in Section 3.1 that transforms a text document into a pseudo-summarization dataset. We select the pseudo-summary using the combination of ROUGE and FactCC. Here, sentence A is selected as the pseudo-summary, and we mask this sentence in the original text to create the pseudo-document. During fine-tuning (b), the connector (i) simulates the factGSG task by appending the same mask token used in (a) to the input document, so that we have the same setup in both training stages. Then, corrector (ii) removes hallucinations (highlighted in red) from the summary. Finally, contrastive learning in (iii) encourages the model to prefer the corrected summary over the perturbed summary.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
like XSum that contains hallucinations on 70% of the summaries (Maynez et al., 2020).
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
To this end, we present three complementary fine-tuning modules, illustrated in Figure 1b. Each component addresses different parts of the downstream task and collaboratively ensures factuality throughout the fine-tuning stage.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
The GSG objective enables faster and better adaptation during fine-tuning by simulating the downstream task (Zhang et al., 2020). However, there still exists a gap between pre-training and finetuning: GSG is a masked sentence prediction task, but downstream summarization does not make use of the mask token. Thus, we simply insert the mask token into the input document of the downstream dataset, so as to simulate what the model expects during pre-training. This can be seen as a form of prompting, which helps us to elicit the factuality knowledge of the pre-trained models. We insert the mask token between sentences, and the best position is determined by evaluating the summarization performance on the validation set. We report the best position of the mask token and discuss the similarity to prompting in Appendix C.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
The corrector removes hallucinations in the reference summaries so that such examples can be used during training without contributing to the problem of imitative falsehood. We consider summary entities as hallucinating if the text cannot be matched to one of the document entities. We propose three approaches with varying degrees of aggressiveness w.r.t. the removal of hallucinations and the possibility of generating ungrammatical sentences.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
Replace: Upon qualitative analysis, we discover that some hallucinated entities in the summary are partially present in the documents. The most prominent example is the use of names, where the summary contains the full name of the person while only the first or last name is mentioned in the document, as shown in Figure 2. Given such observation, we propose a method to find a similar entity with the same NER label in the document and use that to replace the original hallucinated entity in the summary. Although this approach cannot correct hallucinations where similar entities are missing in the document, grammaticality is ensured.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
Remove: A more aggressive approach is to remove the hallucinated entities in the training examples. The intuition is that it is often better to not say anything than to say something wrong. We mitigate the problem of creating ungrammatical sentences by removing related words to the removed entities determined by dependency arcs.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
Combined: As a middle ground that ensures no hallucinations are present in the reference summaries while being grammatical when possible, we first replace all possible entities and then apply the remove strategy on the remaining ones.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
We refer the readers to Appendix-B.1 for the details about hallucination detection, as well as the algorithm and discussion of grammatically for the remove method. 
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
Former Arsenal midfielder Arteta has taken up a coaching role at Manchester City.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
Former Arsenal midfielder Mikel Arteta has taken up a coaching role at Manchester City. Combined Former Arsenal midfielder Arteta has taken up a coaching role at Manchester City.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
Former Arsenal midfielder Manuel Pellegrini has taken up a coaching role. Extrinsic
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
Former Arsenal midfielder Wenger has taken up a coaching role. Figure 2: Example output using different strategies of corrector and contrastor. The first two rows show the original document and summary with highlighted entities and their respective labels (date, number, ent). We mark hallucinated entities in the summaries with red, factual entities in document and summary with green and underlined, and removed entities by the corrector with a strikethrough. Perturbed entities by the contrastor are italicized.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
To better distinguish factual summaries from nonfactual ones, we next introduce a contrastive learning objective that encourages the model to prefer factual summaries given the context of the document. We use the document D i as the anchor and only consider the reference summary S i as the positive sample. Then, we create a set of nonfactual summaries N i to form negative pairs following Kryscinski et al. (2020), where we replace factual entities with random entities of the same named entity labels. We experiment with two variants simulating either extrinsic and intrinsic hallucinations. As formulated in Maynez et al. (2020), extrinsic hallucinations refer to entities that are present in the summary but not in the document, whereas intrinsic hallucinations are those that are present in the document but contain inaccurate information or are misplaced. See Appendix B.2 for more details. We stress that we perform contrastive learning between the document and the summary, similar to Lee et al. (2021), instead of between summaries (Cao and Wang, 2021), as it follows closer to the definition of faithfulness -the summary should be generated within the context of the document.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
We use the NT-Xent loss (Chen et al., 2020):
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
and z S j are representation for D i , S i and S j , respectively. We generate z D and z S by performing mean pooling over the last hidden layer of the encoder and decoder output, respectively. sim(·, ·) is the cosine similarity between the representations, and τ is the temperature parameter.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
The final loss is calculated by the sum of the cross-entropy loss L CE and the contrastive loss:   Table 10.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
We describe our experimental setup, and refer to Appendix A for more details.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
We pre-train on the C4 dataset (Raffel et al., 2020), and evaluate our pre-trained model on three downstream abstractive summarization datasets: XSum (Narayan et al., 2018), WikiHow (Koupaee and Wang, 2018), and Gigaword (Rush et al., 2015). XSum is the primary dataset for analysis unless otherwise stated, as most of the factuality works for abstractive summarization evaluate on this dataset. Dataset details are presented in Appendix A.1. We report ROUGE-L (Lin, 2004) to evaluate our generated summaries against the reference. However, we note that this method is not ideal given the presence of hallucinations in the reference summaries (Chen et al., 2021;Maynez et al., 2020): If a more factual model does not produce such hallucinations, the output is scored lower than those that contain the same hallucinations found in the reference.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
To evaluate factuality, there have been many proposed automatic metrics (Durmus et al., 2020;Wang et al., 2020;Scialom et al., 2021). We report FactCC (Kryscinski et al., 2020) and DEP-Entail (Goyal and Durrett, 2021), as they are highly correlated with human judgment of factuality (Pagnoni et al., 2021). For DEP-Entail, we report the tokenlevel and sentence-level error. For FactCC, since the model has been trained to evaluate on single sentences, we calculate the average score across all sentences for each summary.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
To confirm our observation, we conduct human evaluation asking Amazon Mechanical Turk 2 (AMT) to judge the factuality and informativeness of the summaries. We randomly select 100 documents and ask the annotators to check whether each of the generated summaries is factual and informative. Appendix E provides more details.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
For pre-training, we use BART-base's architecture with PEGASUS's SentencePiece (Kudo, 2018) unigram model tokenizer. We first determine the best sentence selection criteria by experimenting with selection criteria that use ROUGE-1, ROUGE-2, and ROUGE-L, as well as combining each with FactCC. To save computation (Lewis et al., 2020;Zhang et al., 2020;Raffel et al., 2020), we pretrain these models on a smaller dataset and fewer training steps. We report the effect of the selection criteria using the normalized ROUGE score and factuality scores over the model that uses ROUGE-1 as the selection criteria. We take the complement of token error and sentence error as token accuracy and sentence accuracy, respectively, to present all metrics where higher is better. Details of pretraining are shown in Appendix A.4.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
Finally, We evaluate our pre-trained model on the three downstream tasks. As baselines, we compare our model to BART-base and PEGASUS*, our variant of the PEGASUS-base as there is no publicly available checkpoint. We train PEGA-SUS* by using the original sentence selection metric (ROUGE-1), and observe higher ROUGE scores on XSum and WikiHow than the ones reported in the original paper. We also compare FACTPEGA-SUS to two summarization models optimized for factuality. DAE (Goyal and Durrett, 2021) uses 2 https://www.mturk.com/   DEP-Entail to mask out the nonfactual tokens during training, and CLIFF (Cao and Wang, 2021) uses contrastive learning between the reference summaries and automatically generated nonfactual summaries. We apply both methods to BART-base. Details are described in Appendix A.5. Figure 3 shows the effect of different sentence selection criteria. Adding FactCC to all three ROUGE-only criteria consistently improves all factuality metrics at the cost of a small decrease in quality. Overall, the selection strategy of combining ROUGE-1 and FactCC achieves the highest FactCC score out of all strategies while maintaining the smallest relative drop in ROUGE.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
We present our full result on the three downstream tasks in Table 1. While the two baseline models achieve similar factuality scores, FACTPEGASUS consistently improves factuality over the two baselines on all three datasets. The largest improvement can be seen for the XSum dataset, where FACT-PEGASUS, compared to BART-base, lowers the token error and sentence error by 51% and 36%, respectively, and increases FactCC by 43% 3 . The same trend but to a lesser degree can also be observed for WikiHow and Gigaword, most notably a 3-point decrease in sentence error for WikiHow and a 2-point increase in FactCC for Gigaword.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
Compared to factuality-aware models, FACTPE-GASUS achieves the highest FactCC on all tasks. Notably, FACTPEGASUS outperforms DAE by 34% on XSum. In terms of DEP-Entail, FACTPE-GASUS outperforms CLIFF on XSum and Wiki-How. We note that DAE is trained using the DEP-Entail metric and thus is not a fair comparison.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
We note that the ROUGE-L scores for FACT-PEGASUS are lower than both baseline models by about 2 points, but we stress that our increase in FactCC is substantially larger than the decrease in ROUGE-L for XSum and Gigaword. The negative relationship between factuality metrics and ROUGE is also reported in prior works (Chen et al., 2021;Kryscinski et al., 2019). For example, finetuning BART on a subset of XSum (Goyal and Durrett, 2021) improves factuality at the cost of a 6-point drop in ROUGE-L 4 , which is triple the amount of decrease observed for our model.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
Human Evaluation results are shown in Table 2. The result agrees with our observation on automatic factuality metrics, as FACTPEGASUS produces significantly more factual summaries than the BART-base, and PEGASUS*, and CLIFF. We achieve moderately significantly better summaries (p = 0.055) than DAE. Although, FACTPEGA-SUS achieves low informativeness, we find no statistical significant difference between our model and other models (p > 0.15).  Table 3: Fine-tuning ablation on XSum. We present our pre-trained model factGSG fine-tuned without any of our proposed components, and adding different strategies of corrector and contrastor. We then combine the best of the two modules (corrector combined and contrastor intrinsic), and finally add the connector to form the final model, which we copy from  
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
We present ablation studies of our proposed methods in Table 3. We first compare the performance of different strategies for the corrector and contrastor. For corrector, the level of aggressiveness in correcting hallucinations has a positive relationship with factuality metrics but a negative relationship with ROUGE-L. Although the remove method achieves the highest FactCC score, the combined method further lowers the token and sentence error while achieving relatively high ROUGE-L and FactCC. For contrastor, simulating intrinsic errors, which creates more challenging negative samples, provides better factuality results than simulating extrinsic ones. Finally, we show the additive gain in combining the best corrector and contrastor, as well as adding the connector to form the final model. We report the same ablation studies for Gigaword and Wikihow in Appendix D.3, and that for PEGASUS* in Appendix D.4.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
With the help of connector proposed in Section 3.2.1, we can explore how knowledge about factuality is transferred to fine-tuning, especially in the zero-shot and few-shot settings 5 .
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
Zero-Shot. We apply the mask token to the best position and directly analyze the performance of the models on the test set. To better understand the effectiveness in transferring knowledge about summarization and factuality from the pre-training objective, we apply the connector to our pretrained model (factGSG+mask) and PEGASUS* (GSG+mask), so that the two models differ only in their pre-training objective. We report the result in Table 4. FactGSG+mask outperforms GSG+mask on all metrics, especially for factuality metrics. Specifically, factGSG+mask lowers the sentence error by 5 points and increases FactCC by about 10 points. This observation confirms that the factGSG objective is more effective at capturing factuality than the original GSG objective.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
Few-Shot. We follow a similar setup in Zhang et al. (2020), where we limit the number of training data to 1, 10, 100, and 1,000, and then fine-tune the model up to 2,000 steps with the patience of 10 epochs for early stopping. We select the checkpoint with the best validation performance.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
We conduct this experiment by comparing FACT-PEGASUS to PEGASUS*, which has been shown for its ability to transfer with as little as 100 training examples (Zhang et al., 2020). In addition, we report the performance of factGSG+mask to understand how the the model is affected without explicitly ensuring factuality (i.e. without corrector and contrastor). As shown in Figure 4, connector allows the model to better make use of the knowledge of pre-training and produces highquality summaries, as both FACTPEGASUS and factGSG+mask produces a ROUGE-L score comparable to PEGASUS* trained with 1000 examples.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
In terms of factuality, we notice that with just 10 examples, PEGASUS* starts to degrade in factuality, which also applies to the factGSG+mask model. However, FACTPEGASUS demonstrates an opposite trajectory: Sentence error decreases with more training data, and FactCC remains about the same score. This indicates that factual behavior is prone to be overwritten when factuality is not ensured explicitly, and thus calls for the importance of the corrector and contrastor.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
To see whether the factuality degradation observed in few-shot experiment also applies to the full fine- tuning process, we extend our analysis by studying the factuality dynamics, similar to Goyal et al. (2022). The authors observe an increase in sentence errors with the BART model during finetuning, and we analyze whether similar factuality degradation occurs for FACTPEGASUS. We save checkpoints of our models every 10% of the total training steps, and evaluate the models on all three factuality metrics. Figure 5 shows the factuality dynamics during fine-tuning. We notice that the degradation occurs for both models but at a different degree. The token and sentence error for BART-base increase by 2 and 8 points, respectively. However, factuality for FACTPEGASUS remains similar, with only an increase of 1 point for token error and 4.8 points for sentence error. The degradation is only about half of what is observed with BART-base, indicating that FACTPEGASUS is better at avoiding learning nonfactual behaviors.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
Lastly, we wish to understand whether our proposed method is effectively improving factuality without relying on the increase in extractiveness. To this end, Ladhak et al. (2021) introduces a faithfulness-abstractiveness trade-off curve to measure the faithfulness given the model's extractiveness. The authors kindly provided the same set of examples for Gigaword and AMT template for calculating the faithfulness score.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
We show our result on Gigaword in Figure 6. We include the result of their proposed Selector and previous works, including Loss Truncation (Kang and Hashimoto, 2020) and DAE (Goyal and Dur-rett, 2021). We note that the baseline models increase factuality but mostly due to an increase in extractiveness and thus fall below the curve. In contrast, FACTPEGASUS lies above the line, indicating that we are effectively increasing factuality without relying much on becoming more extractive.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
In this work, we proposed FACTPEGASUS, a model for abstractive summarization consisting of factuality-aware pre-training and modules for ensuring factuality during fine-tuning. We demonstrated the effectiveness of our model at improving factuality on three downstream abstractive summarization datasets, confirmed by our human evaluation. Our analysis showed that our proposed factuality-aware pre-training objective is effective at capturing knowledge of factuality compared to the original objective and that our fine-tuning modules reduce the factuality degradation observed with current models. We finally showed that improvement in factuality is not solely explained by the increase of extractiveness.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
Our work aims at reducing the risk of generating hallucinations, and even possibly misinformation, for abstractive summarization models so that such models can be used safely for real-world applications. While we demonstrate that we can alleviate this problem, we stress that there is still a long way to go for improving factuality. Thus, we stress that such models should be used with caution for real-world applications. 
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
A.1 Datasets
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
Following PEGASUS, we pre-train on the C4 dataset, a large collection of documents from Common Crawl. We evaluate our pre-trained model on three downstream abstractive summarization datasets: XSum, WikiHow, and Gigaword. XSum is a collection of articles from the British Broadcasting Corporation, Gigaword is a large collection of news articles headlines, and WikiHow consists of how-to articles. We show the dataset statistics for pre-training and fine-tuning in Table 5, where we present the number of examples in the training, validation, and test splits. We also show the number of examples corrected using the replace and remove method. All datasets are from datasets (Lhoest et al., 2021).
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
We use the ROUGE package provided by transformers (Wolf et al., 2020). We follow the instructions provided by the authors of the factuality metrics to set up and run their code. We report all scores of our models from single runs.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
We use transformers library for the training script and the checkpoints of the pre-trained models. We use the default setting, including the AdamW optimizer and the linear rate scheduler. We also use mixed precision for both pre-training and finetuning the models. We conduct our experiments on the RTX A6000 GPU with 48GB memory and the A100 GPU with 40GB memory. BART-base model has 139M parameters, and PEGASUS* and FACTPEGASUS have 175M parameters.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
Model Architecture. We use the same architecture as BART-base. Specifically, the model has L = 6, H = 768, F = 3072, A = 12, where L is the number of layers, H is the hidden size, F is the dimension for feed-forward layer, and A is the number of self-attention heads. We use the Sen-tencePiece (Kudo, 2018) unigram model tokenizer from PEGASUS with a vocabulary size of 96,103.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
Sentence Selection Criteria. Before pretraining the full model, we first determine the best sentence selection criteria that produces more factual summaries with comparable quality. We experiment with sentence selection criteria that use ROUGE-1, ROUGE-2, and ROUGE-L, as well as combining each with FactCC. To understand the effect of the pre-training objective on factuality directly, we evaluate the performance on the XSum dataset without applying any of our proposed fine-tuning modules. Following Zhang et al.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
(2020), we report the models' relative performance to the base model, which only uses ROUGE-1 as the selection criteria. We use the normalized ROUGE F1 scores 1 3 ( R1 R1 base + R2 R2 base + RL RL base ), where R1 base , R2 base , and RL base are the ROUGE F1 scores of the base model. We similarly report the factuality metrics by normalizing each score by that of the base model. We take the complement of token error and sentence error as token accuracy and sentence accuracy, respectively, to present all metrics where higher is better.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
Similar to previous works (Lewis et al., 2020;Zhang et al., 2020;Raffel et al., 2020) that save computational resources when selecting strategies for pre-training, we pre-train these model on the realnewslike subset of the C4 dataset with less steps.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
Pre-training Details. We use a learning rate of 1e-4, a weight decay of 0.01, and set the maximum number of input tokens to be 512 and a maximum number of output tokens to be 256. We use a batch size of 256. We pre-train the full model for 750,000 steps with a warm-up of 20,000 steps, and only pretrain the smaller models for the sentence selection criteria experiment for 250,000 steps. Pre-training the smaller models takes 30 hours, and pre-training the full model takes 90 hours.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
Calculating FactCC Score. In practice, running FactCC on each sentence-document pair of the pretraining data is expensive. Thus, we opt to only calculate the FactCC score for the top 5 sentences  according to the ROUGE score between the sentence and the rest of the document.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
For all datasets, we use a label smoothing of 0.1. For decoding, we use a beam size of 6 for all datasets. Task-specific hyper-parameters are shown in Table 6. Fine-tuning on XSum and WikiHow takes 8 hours, and fine-tuning on Gigaword takes 11 hours. Decoding on XSum and Gigaword takes half an hour, while decoding WikiHow takes an hour. We use 5 negative examples for the contrastor and set λ to 5 when calculating the combined loss. We set the temperature τ to 0.05. For fine-tuning DAE and CLIFF, we follow the authors' instructions and fine-tune BART-base with their respective code and hyper-parameters. For WikiHow and Gigaword, we use the same hyperparameters as above.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
We use spaCy's NER model 6 to find entities in the document and summary. Entities in the summary sentence are considered nonfactual if no matching document entities with the same string are found. We have previously experimented with the additional requirement of matching entity type similar to Kryscinski et al. (2020), but we find that this constraint unintentionally causes some correct entities to be considered hallucinating, leading to unnecessarily less informative summaries when removed. Given hallucinated entities, we can perform either replace or remove operations. For replace, we find document entities whose words are all contained in the selected entity.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
For the remove method, we need to make sure to also remove any related words. We use spaCy's 6 We use the en_core_web_trf model. dependency parser to systematically remove those. The algorithm is as follows: We first add all the tokens in the selected hallucinated entity to the list of tokens to remove. Then, we recursively find all parents that contain the dependency relation of pobj and prep without any other children and add those to the tokens to remove. Finally, we add all children that do not have the label compound, relcl, and fixed. The final set of words will then be removed in the summary sentence.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
We qualitatively observe that this approach can cover most of the edge cases that would otherwise result in ungrammatical sentences. Nevertheless, this method is not perfect. We include some sample output with the remove method in Figure 7. The algorithm is good at removing entities and related words, such as prepositions, as illustrated in example 1, 3, and 5. However, we observe that it will create ungrammatical sentences when the hallucinated entity is the subject (example 2), or the object of a transitive verb (example 6).
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
We leave exploration with the best systematic correction algorithm or models for future work.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
Similar to Kryscinski et al. (2020), we generate hallucinated summaries by performing entity perturbation on the original summaries. We find entity candidates using the NER labels and sort them into three categories: We include MONEY, QUAN-TITY, and CARDINAL as number, DATE and TIME as date, and all other labels as named entities. We randomly select a factual entity in the summary and replace it with an entity belonging to the same category.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
For extrinsic hallucinations, we sample candidates of the same category from the training corpus but exclude those present in the document. For the intrinsic case, we select to consider the entities from the document. The number of negative examples for all tasks is 5.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
Learning rate Num Steps Warmup Batch size Max Input tokens Max Target tokens   XSum  3e-5  15k  500  256  512  64  WikiHow  3e-5  15k  500  256  512  256  Gigaword  3e-5  50k  2000  256 128 32  
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
This mask-token fine-tuning technique can be seen as a form of prompting, where we elicit our desired faithful abstractive summarization behavior from the pre-trained model directly. Specifically, we consider this as null-prompting (Logan et al., 2021), where using the mask token as the prompt can achieve competitive results with manually engineered prompts. Conveniently, since the mask token during pre-training already serves as a placeholder of where the summary sentence should be generated, it naturally serves as a valid prompt. Figure 1b shows an example of adding the mask token before the first sentence and thus creating a similar setup for pre-training.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
We first need to determine the best position of mask token, as discussed in Section 3.2.1, where we insert the mask token before the ith sentence of the document, where i = 1, 2, ..., 6, and select the best position that achieves the highest ROUGE score on the dev collection. We report ROUGE score of all positions in Table 7 for the three datasets. Interestingly, we observe that the best mask token position for all datasets is before the first sentence. This agrees with the dataset generation of XSum: the summary is taken from the first sentence of the original article. For Gigaword, there is not a change after the first sentence, since the document only consists of a single sentence.   Table 9: Full Result of zero-shot and few-shot experiments.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
We report the full result for the sentence selection criteria in Table 10. Surprisingly, each sentence selection criteria that uses FactCC excels in one specific factuality metric: R1+FactCC is best at FactCC, R2+FactCC is best at sentence error, and RL+FactCC is best for token error.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
We present additional results of the zero-shot and few-shot experiments here.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
Zero-shot We first report the reference-based result of the two baseline models and FACTPEGA-  Table 10: Full result of pre-trained models with different sentence selection criteria shown in Figure 3. We denote the criteria with FactCC with (+FC). Table 8. Due to the mismatch of pretraining and fine-tuning, we observe that both baseline models perform much worse than their result when fully trained. However, with the help of the connector, we observe 11.5 ROUGE-1 points increase for our model compared to the baseline models, and almost four times and double the score for ROUGE-2 and ROUGE-L, respectively.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
Few-shot We show FACTPEGASUS's full result of the few-shot experiment in Table 9.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
We report ablation of each fine-tuning components on Gigaword and Wikihow. The result can be found in Table 11. We observe similar trend as Table 3, where each component improves the performance. For WikiHow and Gigaword, the extrinsic method for contrastive learning perform the best. We think that this is due to the fact that the two tasks do not contain rich entities in the document, and thus require introduction of additional entities from the training corpus.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
We similarly perform the same ablation using the PEGASUS* model, which we present in Table 12.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
We observe similar trend as Table 3. We note that using our pre-trained model factGSG achieves better factuality than PEGASUS* in each setting.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
To ensure high-quality annotations, we select the workers from the United States and have more than 10,000 number of HITS approved as well as an approval rate greater than 98%. In addition, we also create a qualification test where we rate the factuality of the selected generated summaries. Such examples include cases where some summaries hallucinate the first name of a person, which the workers should mark them as not factual. Only workers with the correct annotation can perform the actual task.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
To avoid giving too much text to the workers, we select the most important sentences and replace the less relevant sentences with an ellipsis. For each of the summaries, we select the ten most relevant sentences from the document by cosine similarity of the sentence embedding using SentenceTransformer 7 (Reimers and Gurevych, 2019). We combine and show all the selected relevant sentences from each summary. Since the summaries are similar, we see a large overlap of the relevant sentences.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
We give the following prompt, which we modify from Dreyer et al. (2021): • consistency/factuality: Please avoid using general knowledge, and only consider it in the context of the provided document. Select not consistent if facts in the summary are not supported by the document, such as cases like these:
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
1. The summary contradicts the information in the document. The summary might say "A fire broke out in Seattle", but a document says it broke out in Portland. Or the summary might say "the Republicans won the election", but the document indicates the Democrats won instead 2. The summary adds (hallucinates) a fact that is not mentioned anywhere in the document. For example, the summary might say that "A fire broke out at 2 am", but the document doesn't mention the time when the fire broke out.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
• Informativeness: Please select informative if the summary expresses the main points of the document. Summary should contain relevant and important information and few unimportant details. If you select the summary to be not consistent with the document, please only consider the consistent information when evaluating this category.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
The order of the summary is randomly shuffled. Each task consists of three unique workers, where we take the mean as the scores for this document. The final score is the mean factuality score across all documents. The average time for each task is 7 We use the all-mpnet-base-v2 model.   Table 12: Fine-tuning ablation on XSum using PEGA-SUS*. We combine the modules by using the corrector combined and contrastor intrinsic. We name the model with all three components as PEGASUS* full.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
around 3 minutes and we pay 0.6 USD per task, hence an hourly rate of ≥ $12 per hour. We use boostrap test (Efron and Tibshirani, 1993) to determine statistical significance between two summaries with p < 0.05.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
We include some sample outputs in Figure 8 and Figure 9.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
Example 1 (XSum) Summary: Tap water in 80,000 homes in Lancashire has been declared safe to drink, after the discovery of a parasite at a treatment works left residents boiling water for three weeks. Remove: Tap water in homes has been declared safe to drink, after the discovery of a parasite at a treatment works left residents boiling water .
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
Example 2 (XSum) Summary: Great Britain's Becky James won her second Olympic silver of Rio 2016 by finishing second in the women's sprint. Remove: Won her second Olympic silver of Rio 2016 by finishing second in the women's sprint.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
Summary: Use a "magic eraser.", Try a regular eraser. Make an alcohol solution. Apply a dry erase marker over the permanent marker. Try nail polish remover or acetone on plastic.Acetone and nail polish removers containing acetone can strip away most paints and dyes, and the dye in Sharpies is usually weak against it. Remove: Use a "magic eraser.", Try a regular eraser. Make an alcohol solution. Apply a dry erase marker over the permanent marker. Try nail polish remover or acetone on plastic.Acetone and nail polish removers containing acetone can strip away most paints and dyes, and the dye is usually weak against it.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
Summary: Call 911 if you think you are in danger. Cut through your denial. Do not confront your stalker. Tell other people. Keep a record of stalking activities. Get a restraining order. Be prepared to defend yourself. Change your daily routine. Change your passwords. Use anti-virus software. Clean up your social network. Make your online materials private. Notice signs of control in your relationship. Make your break up clear. Cut off contact. Get back your keys. Remove: Call if you think you are in danger. Cut through your denial. Do not confront your stalker. Tell other people. Keep a record of stalking activities. Get a restraining order. Be prepared to defend yourself. Change your routine. Change your passwords. Use anti-virus software. Clean up your social network. Make your online materials private. Notice signs of control in your relationship. Make your break up clear. Cut off contact. Get back your keys.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
Summary: xinhua summary of asia-pacific stocks news on tuesday feburary ## Remove: summary of stocks news on tuesday ## Example 6 (Gigaword) Summary: cuba urges eu to drop its common position Remove: cuba urges to drop its common position Figure 7: Summaries changed using the corrector. We mark hallucinated entities in the summaries with red.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
Article: The Scots started their Six Nations campaign with a thrilling first win over Ireland in four years. They beat France for the first time in 10 years last season at home, but have lost on their past nine trips to Paris. "It is a long time ago since we won there but we understand there is a reason for that," said Laidlaw. "France are a good team and it is a hard place to win. We are under no illusions this week -we need to be on top form to get something out of the game. "We'll need accuracy and we need to be physical and aggressive because the French are big men. But we need to be subtle and clever in our attack. "If we do that, and move France about and play for 80 minutes, I would back our fitness against them. But we obviously need to be in the game right to the death and not let France get any momentum." Media playback is not supported on this device France have not finished in the top half of the Six Nations since 2011, but impressed in narrow defeats by New Zealand and Australia in the autumn, and came close to a shock victory against England at Twickenham in their Six Nations opener. "They have improved massively under [coach] Guy Noves and have not been far away from any team they have played," Laidlaw noted. "They have some really dangerous strike runners throughout the team. "As much as we need to understand why we beat them last year, we need to develop that, be clever and smart and be able to score points." We need to keep doing what we are doing well, keep our skills high and if we do that again this weekend, we are going to put ourselves in a strong position. "The start of the game is massively important, whether it is our kick-off, or or us receiving. It is very important psychologically for them as it is for us. "If we can start the game playing in the right areas of the field for the first 20 minutes and not muck about too much in our own half, hopefully we will put them under pressure and upset the crowd a little bit." Media playback is not supported on this device Reference: Captain Greg Laidlaw says Scotland must be "subtle and clever" as well as match France physically if they are to secure a first win in Paris since 1999. Bart-base: Scotland head coach Greig Laidlaw says his side are "under no illusions" when they face France at Twickenham on Saturday. PEGASUS*: Scotland coach Steve Laidlaw has warned France that they need to be "smart and smart" in their attack if they are to win their first Six Nations game of the season against the French in Paris on Saturday. CLIFF: Scotland head coach Mark Laidlaw says his side need to be " subtle and clever" in their attack against France in Paris onSaturday. DAE:Scotland must be " subtle and clever" in their attack when they face France at Twickenham on Saturday, says head coach Gordon Laidlaw. FACTPEGASUS: Scotland captain Laidlaw says his side need to be "subtle and clever" in their attack if they are to beat France.
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
Article: the investigating magistrate and prosecutor handling the case involving the french association , zoe ' s ark , arrived tuesday in the eastern town of abeche where they met the ### children the association was attempting to fly to france , according to reports .
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
Reference: chadian court officials go to UNK to investigation into children 's case Bart-base: zoe's ark investigating magistrate arrives in abeche PEGASUS*: prosecutor in zoe's ark case arrives in abeche CLIFF: prosecutor meets zoe 's ark children DAE: zoe's ark investigating magistrate prosecutor arrive in abeche FACTPEGASUS: prosecutor arrives in abeche to investigate french association 
Improving the Faithfulness of Abstractive Summarization via Entity Coverage Control
Abstractive summarization aims to generate a compact and fluent summary that preserves the most salient content of the source document. Recent advances in pre-trained language models (Devlin et al., 2018;Liu and Lapata, 2019; have led to improvements in the quality of generated summaries.
Improving the Faithfulness of Abstractive Summarization via Entity Coverage Control
However, one prominent limitation of existing abstractive summarization systems is the lack of faithfulness of generated outputs. Faithful summaries should only contain content that can be derived from the source document instead of hallucinated or fabricated statements. Summary hallucination could be categorized by the information source as intrinsic and extrinsic hallucinations. Cao et al. (2018); Kryściński et al. (2019) showed that about 30% of the summaries generated by seq2seq models suffer from the hallucination phenomenon at  either the entity level or the summary level. Table  1 shows an example of a model generated summary with hallucinated entities. The BBC article discusses a teenage science competition streamed on the Youtube website, while a BART-based summarizer makes up the term 'Gumtree' instead. Such hallucinations may cause factual errors and hinder the practical use of summarization models.
Improving the Faithfulness of Abstractive Summarization via Entity Coverage Control
Faithfulness and factuality in abstractive summarization has received growing attention from the NLP community (Kryscinski et al., 2020;Goyal and Durrett, 2021;Narayan et al., 2021). Recent works have attempted to address the hallucination problem at the entity level by reducing hallucinated entities during generation. Chen et al. (2021) proposed a post-processing method, which replaces the hallucinated entities in the generated outputs with the same type entities in the source document. However, it introduces additional errors to the summary and increases the intrinsic hallucination. Nan et al. (2021) proposed to address entity hallucination by filtering the training data and multi-task learning with summary-worthy named-entities classification. However, the method sacrifices part of the training data and decreases the quality of the summary.
Improving the Faithfulness of Abstractive Summarization via Entity Coverage Control
To address the above issues, we propose to solve entity hallucination by guiding the model learning process with entity control code (ECC) He et al., 2020;Fan et al., 2017). We utilize the entity coverage precision between the training document and its reference summary as faithfulness guidance and prepend it to the corresponding document in the training phase. Then, we prepend faithful control code during inference and reduce hallucinated entities effectively without decreasing the fluency and salience of generated summaries according to our experimental results. In addition, we extend control code to a Wikipediabased intermediate fine-tuning model, which generates faithful and salient summaries across domains in the zero-shot setting. We validate our methods on three benchmark datasets across different domains, and experimental results demonstrate the effectiveness of our methods.
Improving the Faithfulness of Abstractive Summarization via Entity Coverage Control
Let D = {(d 1 , s 1 ), (d 2 , s 2 ), ..., (d n , s n )} denote a dataset composed of n document and summary pairs. During inference phase, a seq2seq model generates summary hypothesis h i for a given document d i by computing the probability p θ (h i |d i ). The generated summary h i is expected to be faithful, which means all the information in h i should be entailed by the source document d i .
Improving the Faithfulness of Abstractive Summarization via Entity Coverage Control
Following (Nan et al., 2021), we quantify entitylevel hallucination with entity coverage precision prec en . It approximates the faithfulness by measuring the ratio of the named entities in the summary that are coming from the source document. Formally, it is defined as:
Improving the Faithfulness of Abstractive Summarization via Entity Coverage Control
where N (t) represents the set of all named entities found in a given input text t.
Improving the Faithfulness of Abstractive Summarization via Entity Coverage Control
Figure 1 shows our entity coverage control method. We generate a control code C i for each training document and reference summary pair (d i , s i ) so the seq2seq model generates a summary conditioned on both the source document d i and its control code C i , which is represented as p θ (h i |d i , C i ).
Improving the Faithfulness of Abstractive Summarization via Entity Coverage Control
We first compute entity coverage precision prec en for each document and reference summary pair (d i , s i ) in the training set D. Then, we quantize prec en into k discrete bins, each representing a range of entity faithfulness. These bin boundaries are selected to ensure that each bin contains roughly the same number of training examples to avoid data imbalance. We then represent each bin by a special token control code C i and add all these special tokens {C 1 , C 2 , ..., C k } to the input vocabulary of our seq2seq model.
Improving the Faithfulness of Abstractive Summarization via Entity Coverage Control
During training, we prepend the corresponding pseudo label C i to the input document as control code. The seq2seq model is now conditioned on both the source document d i and its control code C i , so it could learn different faithful level generation patterns from the control codes. Then during inference, we prepend the high faithfulness control code C k to all documents in the test set and generate faithful summaries by p θ (h i |d i , C k ).
Improving the Faithfulness of Abstractive Summarization via Entity Coverage Control
Large pre-trained language models (Devlin et al., 2018; perform poorly in the zero-shot summarization setting since sentence salience information is not learned through pretraining tasks (Zhang et al., 2020b). Thus, we propose a controllable generalized intermediate finetuning for zero-shot summarization.
Improving the Faithfulness of Abstractive Summarization via Entity Coverage Control
We first generate pseudo document summary pairs from Wikipedia article dump with similar summary length (n), document length (m) and abstractiveness (a) to the target datasets following Wikitransfer (Fabbri et al., 2021). Instead of training different models for different target datasets as in WikiTransfer, we propose a unified model that generalizes well across different domains. Assume we have l target-specific pseudo training subsets {D 1 (n 1 , m 1 , a 1 ), ..., D l (n l , m l , a l )}, we give each subset another special token E i as a pseudo label to represent the target-specific pattern and also add all these special tokens {E 1 , E 2 , ..., E l } to the input vocabulary of the seq2seq model. In the training phase, we prepend the corresponding target code E i to the document, and a summary is generated conditioned on both the source document d i and its target control code E i , which is represented as p θ (h i |d i , E i ). This allows for control over the domain and generation style of gen-   erated summaries by prepending different domain control codes during inference. The control codes are also stackable, so we can stack the target control with entity coverage control for faithful zeroshot summarization, which could be denoted as
Improving the Faithfulness of Abstractive Summarization via Entity Coverage Control
3 Experiments
Improving the Faithfulness of Abstractive Summarization via Entity Coverage Control
Datasets, evaluation and implementation: We experiment with three summarization datasets in different domains: news dataset XSum (Narayan et al., 2018), scientific paper dataset Pubmed (Cohan et al., 2018), and dialogue dataset Samsum (Gliwa et al., 2019). We use Entity Precision (Nan et al., 2021) and FEQA (Durmus et al., 2020) to evaluate summary faithfulness and use ROUGE (Lin, 2004) to evaluate the fluency and salience. We also ask expert annotators to perform a human evaluation in both summary faithfulness and quality. We use BART-large as backbone model and set hyperparameter k = 3 for all experiments. The three discrete ECC bins are represented with control codes: <FF-low>, <FF-mid> and <FF-high> respectively. More implementation details are described in Appendix A.
Improving the Faithfulness of Abstractive Summarization via Entity Coverage Control
Baselines: We compare our methods with two state-of-the-art methods in summarization faithful-   ness: (1)Post-processing correction in (Chen et al., 2021) (2)Entity-based data filtering in (Nan et al., 2021) together with original BART. For zero-shot summarization, we compare with state-of-the-art method WikiTransfer (Fabbri et al., 2021). Table 2 shows the performance of ECC in the supervised setting. Compared to the summaries generated by BART, our method increases the entity coverage precision significantly with roughly the same summary quality. Table 3 shows the performance comparison to strong baselines on the XSum dataset. Our methods achieves comparable faithfulness improvements without degrading the summary quality compared to data filtering and post-processing methods. We notice there is a trade-off between entity coverage precision and summary quality in Xsum dataset, which is likely due to the low faithfulness level of the reference summaries of Xsum (Maynez et al., 2020). Table 4 shows the zero-shot summarization results. We notice BART tends to copy from the source document, so it achieves high entity coverage precision (92.61) but low summary quality. In contrast, with our intermediate fine-tuning, BART learns the characteristic of the downstream dataset and achieves a considerable improvement in ROUGE score. Compared to the baseline Wikitransfer, we see improvements in both the en-    Table 5 shows the human evaluation results on the 50 randomly sampled subset of articles from the XSum dataset following the setting of (Chen et al., 2021). Four expert annotators assign each summary output into three faithfulness categories (faithful summary, intrinsic hallucination, extrinsic hallucination) and three summary quality categories (low(1), medium (2), high (3)). Note that a summary may contain both intrinsic and extrinsic hallucinations. As the results show, our ECC model improves the faithfulness of the summaries without degrading summary quality, which agrees with our automatic evaluation results.  
Improving the Faithfulness of Abstractive Summarization via Entity Coverage Control
Does our model generate fewer entities to be safe? One obvious way to get higher entity coverage precision is to avoid generating entities or generating extra non-sense named entities from the source document. We show the distribution of the number of entities in the generated summaries by our model and BART in Fig 2. We see that the two distributions are very similar and have almost the same mean number of entities. As a result, we argue that our method doesn't under-generate nor over-generate entities from the source document, and we don't need to separately control the entity compression rate.
Improving the Faithfulness of Abstractive Summarization via Entity Coverage Control
We also study the effect of decoding with different control codes. We prepend different entity coverage control codes during inference on the XSum test set. As shown in Table 6, our model still generates reasonable summaries when inferred with low and medium control codes. We notice that summaries inferred with low control codes have higher ROUGE scores, which agrees with the trade-off described earlier.
Improving the Faithfulness of Abstractive Summarization via Entity Coverage Control
Why does BART generate hallucinated tokens? As shown in Table 7, fine-tuned BART generates 'Gary Anderson' according to the context 'Saints captain Anderson' , which is erroneous since the actual captain is 'Steven Anderson'. Language models contain abundant relational knowledge from pre-training data and could be extracted by masked text filling (Petroni et al., 2019). Similarly, we insert a mask token before 'Anderson' and probe untuned BART to fill the masked tokens. BART generates 'Paul Anderson' (actor) when only given the first sentence context. When given the whole news article, BART learns the context is sports-related and generates famous athletes 'Craig Anderson' (hockey athlete) and 'Gary Anderson' (football athlete) according to its pre-trained prior knowledge. The ground truth 'Steven Anderson' appears much less frequent during pre-training, so BART has a low probability of generating it correctly. We observe the same for ground truth 'Rob Kiernan', which probably appears less frequently in BART's pre-training corpus.
Improving the Faithfulness of Abstractive Summarization via Entity Coverage Control
The faithfulness and factuality in abstractive summarization has received growing attention by the summarization community recently (Kryscinski et al., 2020;Cao et al., 2018;Goyal and Durrett, 2021). Maynez et al. (2020) categorized hallucinations by the information source as intrinsic and extrinsic hallucinations. Researchers have turned to textual entailment (Maynez et al., 2020), question answering (QA) (Durmus et al., 2020;Wang et al., 2020), Natural Language Inference (NLI) (Kryscinski et al., 2020) and entity level precision (Nan et al., 2021) for automatic faithfulness evaluation. To improve the faithfulness of generated summaries. Cao et al. (2018) proposes a fact-aware summarization model with open information extraction and dependency parse technologies.  uses graph attention to integrate factual relations into the summary generation process. Recent works also focus on addressing entity-level hallucination problems. Chen et al. (2021) proposes a post-processing method to correct hallucinated entities and (Nan et al., 2021) addresses entity hallucination by filtering the training data and multi-task learning. One concurrent work Narayan et al. (2021) incorporates entity chain content planning to guide faithful summary generation.
Improving the Faithfulness of Abstractive Summarization via Entity Coverage Control
The transformer-based seq2seq architecture  currently dominates the stateof-the-art performance in many NLP tasks (Liu and Lapata, 2019;Zhang et al., 2020a;. We use BART  as a backbone for abstractive summarization in this work, but our method is generally appliable for all seq2seq models.
Improving the Faithfulness of Abstractive Summarization via Entity Coverage Control
Our work is also related to controllable abstractive summarization. Liu et al. (2018) controls the summary length by extending a convolutional sequence to sequence model. He et al. (2020) introduces a keyword guided framework for entity-centric, length-controllable summarization and question-guided summarization. Fan et al. (2017) proposes to control the summary generation with a list of desired named entities. Recently, Feng et al. (2021) proposes to use language models to generate pseudo labels to control the generation of dialogue summarization. Our work uses control code to improve summary generation faithfulness and cross-domain generalizability.
Improving the Faithfulness of Abstractive Summarization via Entity Coverage Control
In this paper, we propose ECC to address extrinsic hallucination in abstractive summarization in both supervised and zero-shot settings. Our extensive experiments demonstrate that the proposed method effectively reduces entity hallucination without hurting the quality of the generated summaries.
Improving the Faithfulness of Abstractive Summarization via Entity Coverage Control
In Table 8, we provide several representative examples from XSum dataset. Example 1 (first row) shows how our entity control method gets rid of hallucination terms from BART output. The reference summary here is not faithful since 'Los Angeles' is not covered in the source document. The correction baseline changes 'Los Angeles' to 'Mexico', which is a factual error. In contrast, the ECCoutput is totally faithful to the source document and contains salient information.
Improving the Faithfulness of Abstractive Summarization via Entity Coverage Control
Example 2 (second row) shows the outputs decoded with different control codes during inference. We can see the output decoded with low faithfulness control code is still fluent and reasonable, but contains less faithful entities compared to the output decoded with high faithfulness control code.
Improving the Faithfulness of Abstractive Summarization via Entity Coverage Control
Example 3 (third row) shows an example of factual statement, which is verifiable in the real world independent of the source text. The reference summary uses 'most of Wales' to summarize the county names in the source document. This type of hallucination needs more external knowledge and com-  monsense reasoning to decide its factuality. Our method only focuses on entity level hallucination problems instead.
Improving the Faithfulness of Abstractive Summarization via Entity Coverage Control
Our human evaluation follows the setting of prior work (Chen et al., 2021). We calculate the interannotator agreement with additional annotations from two other experts. We estimate the adjusted mean and 95% confidence interval from the mean and standard deviation. The full results are shown in Table 9.
Enhancing Factual Consistency of Abstractive Summarization
Text summarization models aim to produce an abridged version of long text while preserving salient information. Abstractive summarization is a type of such models that can freely generate summaries, with no constraint on the words or phrases used. This format is closer to human-edited summaries and is both flexible and informative. Thus, there are numerous approaches to produce abstractive summaries (See et al., 2017;Paulus et al., 2017;Dong et al., 2019;Gehrmann et al., 2018).
Enhancing Factual Consistency of Abstractive Summarization
However, one prominent issue with abstractive summarization is factual inconsistency. It refers to the hallucination phenomenon that the summary sometimes distorts or fabricates the facts in the article. Recent studies show that up to 30% of the summaries generated by abstractive models contain such factual inconsistencies (Kryściński et al., 2019b;Falke et al., 2019), raising concerns about the credibility and usability of these systems. 1 We provide the prediction results of all models at https: //github.com/zcgzcgzcg1/FASum/.
Enhancing Factual Consistency of Abstractive Summarization
Real Madrid ace Gareth Bale treated himself to a Sunday evening BBQ... The Welsh wizard was ... scoring twice and assisting another in an impressive victory... Cristiano Ronaldo scored five goals against Granada on Sunday ...
Enhancing Factual Consistency of Abstractive Summarization
... The Real Madrid ace scored five goals against Granada on Sunday. The Welsh wizard was in impressive form for...
Enhancing Factual Consistency of Abstractive Summarization
...Gareth Bale scored five and assisted another in an impressive win in Israel...
Enhancing Factual Consistency of Abstractive Summarization
...Gareth Bale scored twice and helped his side to a sensational 9-1 win. Cristiano Ronaldo scored five goals against Granada on Sunday... Table 1 demonstrates an example article and excerpts of generated summaries. As shown, the article mentions that Real Madrid ace Gareth Bale scored twice and Cristiano Ronaldo scored five goals. However, both BOTTOMUP (Gehrmann et al., 2018) and SEQ2SEQ wrongly states that Bale scored five goals. Comparatively, our model FASUM generates a summary that correctly exhibits the fact in the article. And as shown in Section 4.6.1, our model achieves higher factual consistency not just by making more copies from the article.
Enhancing Factual Consistency of Abstractive Summarization
On the other hand, most existing abstractive summarization models apply a conditional language model to focus on the token-level accuracy of summaries, while neglecting semantic-level consistency between the summary and article. Therefore, the generated summaries are often high in tokenlevel metrics like ROUGE (Lin, 2004) but lack factual consistency. In view of this, we argue that a robust abstractive summarization system must be equipped with factual knowledge to accurately summarize the article.
Enhancing Factual Consistency of Abstractive Summarization
In this paper, we represent facts in the form of knowledge graphs. Although there are numerous Table 2: Example excerpts of an article from CNN/DailyMail and the summary generated by BOTTOMUP. Factual errors are marked in red. The correction made by our model FC are marked in green. efforts in building commonly applicable knowledge graphs such as ConceptNet (Speer et al., 2017), we find that these tools are more useful in conferring commonsense knowledge. In abstractive summarization for contents like news articles, many entities and relations are previously unseen. Plus, our goal is to produce summaries that do not conflict with the facts in the article. Thus, we propose to extract factual knowledge from the article itself.
Enhancing Factual Consistency of Abstractive Summarization
We employ the information extraction (IE) tool OpenIE (Angeli et al., 2015) to extract facts from the article in the form of relational tuples: (subject, relation, object). This graph contains the facts in the article and is integrated in the summary generation process.
Enhancing Factual Consistency of Abstractive Summarization
Then, we use a graph attention network (Veličković et al., 2017) to obtain the representation of each node, and fuse that into a transformer-based encoder-decoder architecture via attention. We denote this model as the Fact-Aware Summarization model, FASUM.
Enhancing Factual Consistency of Abstractive Summarization
In addition, to be generally applicable for all existing summarization systems, we propose a Factual Corrector model, FC, to help improve the factual consistency of any given summary. We frame the correction process as a seq2seq problem: the input is the original summary and the article, and the output is the corrected summary. FC has the same architecture as UniLM (Dong et al., 2019) and initialized with weights from RoBERTa-Large (Liu et al., 2019). We finetune it as a denoising autoencoder. The training data is synthetically generated via randomly replacing entities in the ground-truth summary with wrong ones in the article. As shown in Table 2, FC makes three corrections, replacing the original wrong entities which appear elsewhere in the article with the right ones.
Enhancing Factual Consistency of Abstractive Summarization
In the experiments, we leverage an indepen-dently trained BERT-based (Devlin et al., 2018) factual consistency evaluator (Kryściński et al., 2019b). Results show that on CNN/DailyMail, FA-SUM obtains 0.6% higher fact consistency scores than UNILM (Dong et al., 2019) and 3.9% higher than BOTTOMUP (Gehrmann et al., 2018). Moreover, after correction by FC, the factual score of summaries from BOTTOMUP increases 1.4% on CNN/DailyMail and 0.9% on XSum, and the score of summaries from TCONVS2S increases 3.1% on XSum. We also conduct human evaluation to verify the effectiveness of our models. We further propose an easy-to-compute modelfree metric, relation matching rate (RMR), to evaluate factual consistency given a summary and the article. This metric employs the extracted relations and does not require human-labelled summaries. Under this metric, we show that our models can help enhance the factual consistency of summaries.
Enhancing Factual Consistency of Abstractive Summarization
Abstractive text summarization has been intensively studied in recent literature. Rush et al. (2015) introduces an attention-based seq2seq model for abstractive sentence summarization. See et al. (2017) uses copy-generate mechanism that can both produce words from the vocabulary via a generator and copy words from the article via a pointer. Paulus et al. (2017) et al. (2020) improves factual consistency of generated headlines by filtering out training data with more factual errors. In comparison, our model utilizes the knowledge graph extracted from the article and fuses it into the generated text via neural graph computation.
Enhancing Factual Consistency of Abstractive Summarization
To correct factual errors, Dong et al. (2020) uses pre-trained NLU models to rectify one or more wrong entities in the summary. Concurrent to our work, Cao et al. (2020) employs the generation model BART (Lewis et al., 2019) to produce corrected summaries.
Enhancing Factual Consistency of Abstractive Summarization
Several approaches have been proposed to evaluate a summary's factual consistency (Kryściński et al., 2019a;Goodrich et al., 2019;Maynez et al., 2020). Zhang et al. (2019a) employs BERT to compute similarity between pairs of words in the summary and article. Wang et al. (2020); Durmus et al. (2020) use question answering accuracy to measure factual consistency. Kryściński et al. (2019b) applies various transformations on the summary to produce training data for a BERT-based classification model, FactCC, which shows a high correlation with human metrics. Therefore, we use FactCC as the factual evaluator in this paper.
Enhancing Factual Consistency of Abstractive Summarization
We formalize abstractive summarization as a supervised seq2seq problem. The input consists of a pairs of articles and summaries: {(X 1 , Y 1 ), (X 2 , Y 2 ), ..., (X a , Y a )}. Each article is tokenized into X i = (x 1 , ..., x L i ) and each summary is tokenized into Y i = (y 1 , ..., y N i ). In abstrative summarization, the model-generated summary can contain tokens, phrases and sentences not present in the article. For simplicity, in the following we will drop the data index subscript. Therefore, each training pair becomes X = (x 1 , ..., x m ), Y = (y 1 , ..., y n ), and the model needs to generate an abstrative summarŷ Y = (ŷ 1 , ...,ŷ n ).
Enhancing Factual Consistency of Abstractive Summarization
We propose the Fact-Aware abstractive Summarizer, FASUM. It utilizes the seq2seq architecture built upon transformers (Vaswani et al., 2017). In detail, the encoder produces contextualized embeddings of the article and the decoder attends to the encoder's output to generate the summary.
Enhancing Factual Consistency of Abstractive Summarization
To make the summarization model fact-aware, we extract, represent and integrate knowledge from the source article into the summary generation process, which is described in the following. The overall architecture of FASUM is shown in Figure 1.
Enhancing Factual Consistency of Abstractive Summarization
To extract important entity-relation information from the article, we employ the Stanford OpenIE tool (Angeli et al., 2015). The extracted knowledge is a list of tuples. Each tuple contains a subject (S), a relation (R) and an object (O), each as a segment of text from the article. In the experiments, there are on average 165.4 tuples extracted per article in CNN/DailyMail (Hermann et al., 2015) and 84.5 tuples in XSum (Narayan et al., 2018).
Enhancing Factual Consistency of Abstractive Summarization
We construct a knowledge graph to represent the information extracted from OpenIE. We apply the Levi transformation (Levi, 1942) to treat each entity and relation equally. In detail, suppose a tuple is (s, r, o), we create nodes s, r and o, and add edges s-r and r-o. In this way, we obtain an undirected knowledge graph G = (V, E), where each node v ∈ V is associated with text t(v). During training, this graph G is constructed for each batch individually, i.e. there's no shared huge graph. One benefit is that the model can take unseen entities and relations during inference.
Enhancing Factual Consistency of Abstractive Summarization
We then employ a graph attention network (Veličković et al., 2017) to obtain embedding e j for each node v j . The initial embedding of v j is given by the last hidden state of a bidirectional LSTM  applied to t(v j ). In the experiment, we employ 2 graph attention layers.
Enhancing Factual Consistency of Abstractive Summarization
The knowledge graph embedding is obtained in parallel with the encoder. Then, apart from the canonical cross attention over the encoder's outputs, each decoder block also computes cross-attention over the knowlege graph nodes' embeddings:
Enhancing Factual Consistency of Abstractive Summarization
where {e j } |V | j=1 are the final embeddings of the graph nodes, and {s i } t i=1 are the decoder block's representation of the first t generated tokens.
Enhancing Factual Consistency of Abstractive Summarization
We denote the final output of the decoder as z 1 , ..., z t . To produce the next token y t+1 , we employ a linear layer W to project z t to a vector of the same size of the dictionary. And the predicted distribution of y t+1 is obtained by:
Enhancing Factual Consistency of Abstractive Summarization
During training, we use cross entropy as the loss function L(θ) = − n t=1 y T t log(p t ), where y t is the one-hot vector for the t-th token, and θ represent the parameters in the network.
Enhancing Factual Consistency of Abstractive Summarization
To better utilize existing summarization systems, we propose a Factual Corrector model, FC, to improve the factual consistency of any summary generated by abstractive systems. FC frames the correction process as a seq2seq problem: given an article and a candidate summary, the model generates a corrected summary with minimal changes to be more factually consistent with the article.
Enhancing Factual Consistency of Abstractive Summarization
While FASum has a graph attention module in the transformer, preventing direct adaptation from pre-trained models, the FC model architecture adopts the design of the pre-trained model UniLM (Dong et al., 2019). We initiailized the model weights from RoBERTa-Large (Liu et al., 2019). The finetuning process is similar to training a denoising autoencoder. We use back-translation and entity swap for synthetic data generation. For example, an entity in the ground-truth summary is randomly replaced with another entity of the same type from the article. This modified summary and the article is sent to the corrector to recover the original summary. In the experiments, we gener-ated 3.0M seq2seq data samples in CNN/DailyMail and 551.0K samples in XSum for finetuning. We take 10K samples in each dataset for validation and use the rest for training.
Enhancing Factual Consistency of Abstractive Summarization
During inference, the candidate summary from any abstractive summarization system is concatenated with the article and sent to FC, which produces the corrected summary.
Enhancing Factual Consistency of Abstractive Summarization
We evaluate our model on benchmark summarization datasets CNN/DailyMail (Hermann et al., 2015) and XSum (Narayan et al., 2018). They contain 312K and 227K news articles and humanedited summaries respectively, covering different topics and various summarization styles.
Enhancing Factual Consistency of Abstractive Summarization
We use the Huggingface's (Wolf et al., 2019) implementation of transformer in BART (Lewis et al., 2019). We also inherit their provided hyperparameters of CNN/DailyMail and XSum for the beam search. The minimum summary length is 56 and 11 for CNN/Daily Mail and XSum, respectively. The number of beams is 4 for CNN/DailyMail and 6 for XSum.
Enhancing Factual Consistency of Abstractive Summarization
In FASUM, both the encoder and decoder has 10 layers of 10 heads for attention. Teacher forcing is used in training. We use Adam (Kingma and Ba, 2014) as the optimizer with a learning rate of 2e-4.
Enhancing Factual Consistency of Abstractive Summarization
The bi-LSTM to produce the initial embedding of graph nodes has a hidden state of size 64 and the graph attention network (GAT) has 8 heads and a hidden state of size 50. The dropout rate is 0.6 in GAT and 0.1 elsewhere.
Enhancing Factual Consistency of Abstractive Summarization
We use the subword tokenizer SentencePiece (Kudo and Richardson, 2018). The dictionary is shared across all the datasets. The vocabulary has a size of 32K and a dimension of 720.
Enhancing Factual Consistency of Abstractive Summarization
The correction model FC follows the UniLM (Dong et al., 2019) architecture initialized with weights from RoBERTa-Large (Liu et al., 2019). We fine-tune the model for 5 epochs with a learning rate of 1e-5 and linear warmup over the one-fifths of total steps and linear decay. During decoding, it uses beam search with a width of 2, and blocks trigram duplicates. The batch size during finetuning is 24. More details are presented in the Appendix.
Enhancing Factual Consistency of Abstractive Summarization
To evaluate factual consistency, we re-implemented and trained the FactCC model (Kryściński et al., 2019b). The model outputs a score between 0 and 1, where a higher score indicates better consistency between the input article and summary. The training of FactCC is independent of our summarizer so no parameters are shared. More details are in the Appendix.
Enhancing Factual Consistency of Abstractive Summarization
We also employ the standard ROUGE-1, ROUGE-2 and ROUGE-L metrics (Lin, 2004) to measure summary qualities. These three metrics evaluate the accuracy on unigrams, bigrams and the longest common subsequence. We report the F1 ROUGE scores in all experiments. And the ROUGE-L score on validation set is used to pick the best model for both FASUM and FC.
Enhancing Factual Consistency of Abstractive Summarization
The following abstractive summarization models are selected as baseline systems. TCONVS2S  Table 3, our model FASUM 2 outperforms all baseline systems in factual consistency scores in CNN/DailyMail and is only behind UNILM in XSum. In CNN/DailyMail, FA-SUM is 0.6% higher than UNILM and 3.9% higher than BOTTOMUP in factual score. Statistical test shows that the lead is statistically significant with pvalue smaller than 0.05. The higher factual score of UNILM among baselines corroborates the findings in Maynez et al. (2020) that pre-trained models exhibit better factuality. But our proposed knowledge graph component can help the train-from-scratch FASUM model to excel in factual consistency.
Enhancing Factual Consistency of Abstractive Summarization
We conduct ablation study to remove the knowledge graph component from FASUM, resulting in the SEQ2SEQ model. As shown, there is a clear drop in factual score: 2.8% in CNN/DailyMail and 0.9% in XSum. This proves that the constructed knowledge graph can help increase the factual correctess of the generated summaries.
Enhancing Factual Consistency of Abstractive Summarization
It's worth noticing that the ROUGE metric does not always reflect the factual consistency, sometimes even showing an inverse relationship, a phenomenon observed in multiple studies (Kryściński et al., 2019a;Maynez et al., 2020). For instance, although BOTTOMUP has 0.69 higher ROUGE-1 points than FASUM in CNN/DailyMail, there are many factual errors in its summaries, as shown in the human evaluation. On the other hand, to make sure the improved factual correctness of our models is not achieved by simply copying insignificant information from the article, we conduct analysis on abstractiveness in Section 4.6.1 and human evaluation in Section 4.6.3.
Enhancing Factual Consistency of Abstractive Summarization
Furthermore, the correction model FC can effectively enhance the factual consistency of summaries generated by various baseline models, especially when the original summary has a relatively low factual consistency. For instance, on CNN/DM, the factual score of BOTTOMUP increases by 1.4% after correction. On XSum, after correction, the factual scores increase by 0.2% to 3.1% for all baseline models. Interestingly, FC can also boost the factual consistency of our FASUM model. Furthermore, the correction has a rather small impact on the ROUGE score, and it can improve the ROUGE scores of most models in XSum dataset.
Enhancing Factual Consistency of Abstractive Summarization
We check and find that FC only makes modest modifications necessary to the original summaries. For instance, FC modifies 48.3% of summaries generated by BOTTOMUP in CNN/DailyMail. These modified summaries contain very few changed tokens: 94.4% of these corrected summaries contain 3 or fewer new tokens, while the summaries have on average 48.3 tokens.
Enhancing Factual Consistency of Abstractive Summarization
In the appendix of supplementary materials, we show several examples of summaries given by FA-SUM and corrected by FC to demonstrate the improved factual consistency of summarization.
Enhancing Factual Consistency of Abstractive Summarization
It has been shown in Durmus et al. (2020) that less abstractive summaries are more factual consistent with the article. Therefore, we inspect whether our models boost factual consistency simply by copying more portions of the article.
Enhancing Factual Consistency of Abstractive Summarization
On XSum's testset, we compute the ratio of novel n-grams in summaries that do not appear in the article. Figure 2 shows that FASUM achieves the closest ratio of novel n-gram compared with reference summaries, and higher than BOTTOMUP and UNILM. This demonstrates that FASUM can produce highly abstractive summaries while ensuring factual consistency.
Enhancing Factual Consistency of Abstractive Summarization
While the factual consistency evaluator FactCC (Kryściński et al., 2019b) is based on pre-trained models, it requires finetuning on articles and labelled summaries. Furthermore, we empirically find that the performance of FactCC degrades when it is finetuned on one summary dataset and used to evaluate models on another dataset. Therefore, in this subsection, we design an easyto-compute model-free factual consistency metric, which can be used when ground-truth summaries are not available.
Enhancing Factual Consistency of Abstractive Summarization
As the relational tuples in the knowledge graph capture the factual information in the text, we compute the precision of extracted tuples in the summary. In detail, suppose the set of the relational tuples in the summary is R s = {(s i , r i , o i )}, and the set of the relational tuples in the article is R a . Then, each tuple in R s falls into one of the following three categories:
Enhancing Factual Consistency of Abstractive Summarization
3. Miss (M): Otherwise.
Enhancing Factual Consistency of Abstractive Summarization
We define two kinds of relation matching rate   (RMR) to measure the ratio of correct hits:
Enhancing Factual Consistency of Abstractive Summarization
Note that this metric is different from the ratio of overlapping tuples proposed in Goodrich et al. (2019), where the ratio is computed between the ground-truth and the candidate summary. Since even the ground-truth summary may not cover all the salient information in the article, we choose to compare the knowledge tuples in the candidate summary directly against those in the article. An additional advantage of our metric is that it does not require ground-truth summaries to be available.  (Ott et al., 2019). The model predicts the relationship between the article and summary to be one of the following: entailment, neutral and contradiction. We report the ratio of contradiction as predicted by the model in Table 4. As shown, FA-SUM achieves the lowest ratio and FC helps further reducing conflicting facts in generated summaries.
Enhancing Factual Consistency of Abstractive Summarization
We conduct human evaluation on the factual consistency and informativeness of summaries. We randomly sample 100 articles from the test set of CNN/DailyMail. Then, each article and summary pair is labelled by 3 people from Amazon Mechanical Turk (AMT) to evaluate the factual consistency and informativeness. Each labeller gives a score in each category between 1 and 3 (3 being perfect). The kappa-ratio between reviewer scores is 0.32 for   factual consistency and 0.28 for informativeness.
Enhancing Factual Consistency of Abstractive Summarization
Here, factual consistency indicates whether the summary's content is faithful with respect to the article; informativeness indicates how well the summary covers the salient information in the article.
Enhancing Factual Consistency of Abstractive Summarization
As shown in Table 5, our model FASUM achieves the highest factual consistency score, higher than UNILM and considerably outperforming BOTTOMUP. We conduct a statistical test and find that compared with UNILM, our model's score is statistically significant with p-value smaller than 0.05 under paired t-test. In terms of informativeness, our model is comparable with UNILM and outperforms BOTTOMUP. Finally, without the knowledge graph component, the SEQ2SEQ model generates summaries with both less factual consistency and informativeness.
Enhancing Factual Consistency of Abstractive Summarization
To assess the effectiveness of the correction model FC, we conduct a human evaluation of sideby-side summaries. In CNN/DailyMail, we randomly sample 100 articles where the summaries generated by BOTTOMUP are modified by FC. 3 labelers are asked whether the original or the corrected version is factually more correct. We collect all the feedbacks and compute the ratio of judgements for each case. To reduce bias, we randomly shuffle the two versions of summaries. We conduct similar evaluation on UNILM.
Enhancing Factual Consistency of Abstractive Summarization
As shown in Table 6, the corrected summaries are significantly more likely to be judged as more factually correct for both baseline models. For example, 42.3% of the judgements think the corrected summaries are factually more correct, 42.7% think the corrected version neither improves nor worsens the factual consistency, while only 15.0% think that the corrected version becomes worse than the original BOTTOMUP summary. Therefore, FC can help boost the factual consistency of summaries from given systems.
Enhancing Factual Consistency of Abstractive Summarization
Finally, to evaluate the quality of the relation matching rate (RMR), we compute the correlation coefficient γ between the factual score given by human labelers and the RMR value. The result shows that γ = 0.43, indicating observable relationship between RMR and human evaluation results.
Enhancing Factual Consistency of Abstractive Summarization
In this paper, we extract factual information from the article to be represented by a knowledge graph. We then integrate this factual knowledge into the process of producing summaries. The resulting model FASUM enhances the ability to preserve facts during summarization, demonstrated by both automatic and human evaluation. We also present a correction model, FC, to rectify factual errors in candidate summaries. Furthermore, we propose an easy-to-compute model-free metric, relation matching rate, to measure factual consistency based on the overlapping ratio of relational tuples.
Enhancing Factual Consistency of Abstractive Summarization
For future work, we plan to integrate knowledge graphs into pre-training for more accurate and factually consistent summarization. Moreover, we will combine the internally extracted knowledge graph with an external knowledge graph (e.g. Con-ceptNet) to enhance the commonsense capability of summarization models.  
Enhancing Factual Consistency of Abstractive Summarization
For hyperparameter search, we tried 4 layers with 4 heads, 6 layers with 6 heads and 10 layers with 10 heads.
Enhancing Factual Consistency of Abstractive Summarization
There're 108.3M parameters in the FASUM model and it takes 2 hours (CNN/DailyMail) / 0.5 hours (XSum) for 4 v100 GPUs to train 1 epoch. The batch size is set to 48 for both datasets.
Enhancing Factual Consistency of Abstractive Summarization
On validation datasets, FASUM achieves ROUGE-1 41.08%, ROUGE-2 18.35% and ROUGE-L 37.95% on CNN/DailyMail, and it achieves ROUGE-1 30.28%, ROUGE-2 10.09% and ROUGE-L 23.85% on XSum.
Enhancing Factual Consistency of Abstractive Summarization
To automatically evaluate the factual consistency of a summary, we leverage the FactCC model (Kryściński et al., 2019b), which maps the consistency evaluation as a binary classification problem, namely finding a function f : (A, C) −→ [0, 1], where A is an article and C is a summary sentence defined as a claim. f (A, C) represents the probability that C is factually correct with respect to the article A. If a summary S is composed of multiple sentences C 1 , ..., C k , we define the factual score of S as:
Enhancing Factual Consistency of Abstractive Summarization
To generate training data, we adopt backtranslation as a paraphrasing tool. The ground-truth summary is translated into an intermediate language, including French, German, Chinese, Spanish and Russian, and then translated back to English. Together with the original summaries, these claims are used as positive training examples. We then apply entity swap, negation and pronoun swap to generate negative examples (Kryściński et al., 2019b).
Enhancing Factual Consistency of Abstractive Summarization
Following Kryściński et al. (2019b), we finetune the BERT BASE model using the same hyperparameters to finetune FactCC. We concatenate the article and the generated claim together with special tokens [CLS] and [SEP]. The final embedding of [CLS] is used to compute the probability that the claim is entailed by the article content.
Enhancing Factual Consistency of Abstractive Summarization
As shown in Table 7, on CNN/Daily Mail, our reproduced model achieves better accuracy than that in Kryściński et al. (2019b) on the humanlabelled sentence-pair-ordering data (Falke et al., 2019). Thus, we use this evaluator for all the factual consistency assessment tasks in the following. 3  As shown, while baseline systems like BOT-TOMUP and UNILM achieve high ROUGE scores, they are susceptible to factual errors. For instance, in Article 5, both BOTTOMUP and SEQ2SEQ wrongly state that Rickie Fowler accused Alexis. In fact, Alexis, Rickie's girlfriend, was accused by an online hater. In Article 1, UNILM mistakenly summarizes that Arsenal lost 4-1 where in fact Arsenal won 4-1 against Liverpool.
Enhancing Factual Consistency of Abstractive Summarization
In comparison, our proposed fact-aware summarizer FASUM could faithfully summarize the salient information in the article. And it can reorganize the phrasing instead of merely copying content from the article. Table 11 and Table 12 show examples of CNN/DailyMail articles, summaries generated by BOTTOMUP/UNILM and the corrected version by FC. As shown, our correction model can select the wrong entities and replace them with correct ones. For instance, in Article 1, BOTTOMUP's summary states that Rual Castro, who appears elsewhere in the article, is the President of Venezuela, while FC correctly replaces it with Nocolas Maduro. In Article 4, UNILM wrongly attributes the statement to Scott's lawyer (probabily because "Scott" appears closer to the statement in the article), while it was actually said by Slager's lawyer. This error is corrected by FC.
Enhancing Factual Consistency of Abstractive Summarization
Article 1
Enhancing Factual Consistency of Abstractive Summarization
Arsenal target Raheem Sterling is a more talented player than Gunners star Mesut Ozil, according to Michael Owen. The former Liverpool forward made the claim before Brendan Rodgers' men lost 4-1 at the Emirates on Saturday ... Ozil bends home Arsenal's second goal in their 4-1 win against Liverpool at the Emirates...
Enhancing Factual Consistency of Abstractive Summarization
Arsenal are interested in signing Liverpool's Raheem Sterling this summer. The 20-year-old has rejected a new PS100,000-a-week deal at Anfield. Michael Owen believes Sterling is more talented than Mesut Ozil. Adrian Durham: Sterling would be earning the same as Balotelli if he signed PS100,000-a-week deal at Liverpool... Durham: Arsenal only turn it on when the pressure is off. The German scored in Arsenal's 4-1 win against Liverpool on Saturday...
Enhancing Factual Consistency of Abstractive Summarization
Raheem Sterling made the claim before Brendan Rodgers' men lost 4-1 at the Emirates on Saturday. Bt Sport pundit Owen believes there should not be a debate about whether Sterling is more talented than Ozil. The former Liverpool forward is a better player than Mesut Ozil.
Enhancing Factual Consistency of Abstractive Summarization
Michael Owen believes Raheem Sterling is a better player than Mesut Ozil. Owen made the claim before Arsenal lost 4-1 at the Emirates on Saturday. Owen's opinion drew criticism on social media but the former Liverpool forward has refused to back down. Owen said :' Is there a debate? Not even close' CLICK HERE for all the latest Arsenal news.
Enhancing Factual Consistency of Abstractive Summarization
Michael Owen says there should not be a debate about whether Raheem Sterling is more talented than Mesut Ozil. The former Liverpool forward made the claim before Brendan Rodgers' men lost 4-1 at the Emirates on Saturday. Owen repeated his claim that the wantaway forward is a superior player than Ozil made.
Enhancing Factual Consistency of Abstractive Summarization
Michael Owen made the claim before Liverpool's men lost 4-1 at the Emirates on Saturday. Michael Owen believes there should be a debate about whether Sterling is more talented than Ozil. The ex-England striker has refused to back down, even after Sterling's poor display in north London.
Enhancing Factual Consistency of Abstractive Summarization
Yaya Toure was accused by Jamie Carragher of ducking out of the way of Jason Puncheon's free-kick which killed off Manchester City's hopes of salvaging something from Selhurst Park. Toure, who is paid PS200,000-a-week, was part of a five-man wall when Puncheon stepped up to double Crystal Palace's lead...
Enhancing Factual Consistency of Abstractive Summarization
Crystal Palace beat Manchester City 2-1 at Selhurst Park on Monday night. Jason Puncheon's free-kick doubled Palace's lead in the second half. Yaya Toure was accused of ducking out of the way of the winning goal. Sportsmail's Jamie Carragher: 'The rest of them are desperate to be hit with the ball in the face and it's poor from Yaya Toure' Gary Neville: City have a mentality problem... they cannot sustain success. BOTTOMUP Yaya Toure was accused by Jamie Carragher out of the way of Jason Puncheon's free-kick at Selhurst Park on Monday. Toure, who is paid 200,000-a-week, was part of a wall with Crystal Palace's lead. Carragher said the midfielder did not do enough to prevent the winner. UNILM Yaya Toure was accused by Sportsmail's Jamie Carragher of ducking out of the way of Jason Puncheon's free-kick at Selhurst Park on Monday. The midfielder was part of a five-man wall when Puncheon stepped up to double Crystal Palace's lead. Toure is paid 200,000-a-week a by the Premier League champions.
Enhancing Factual Consistency of Abstractive Summarization
Yaya Toure was accused by Jamie Carragher of ducking out of the way of Jason Puncheon's free-kick. Fernandinho and Vincent Kompany look keen to get their head to the ball but Toure appears to duck. Toure paid PS200,000-a-week, was part of a five-man wall when Puncheon stepped up to double Crystal Palace's lead.
Enhancing Factual Consistency of Abstractive Summarization
Jamie Carragher was accused of ducking out of the way of Jason Puncheon's free-kick which killed off Manchester City's chances at Selhurst Park on Monday Night. Toure is paid PS200,000-a-week, was part of a five-man wall when Puncheon stepped up to double Palace's lead. Sportsmail columnist Jamie Carragher feels City midfielder did not do enough to prevent winner.
Enhancing Factual Consistency of Abstractive Summarization
Real Madrid ace Gareth Bale treated himself to a Sunday evening BBQ after ... a sensational victory. The Welsh wizard was ... scoring twice and assisting another in an impressive victory in Israel... he opened the scoring in Real Madrid's impressive 9-1 victory... His goal in the 25th minute may have got the ball rolling ... Cristiano Ronaldo scored five goals against Granada on Sunday ...
Enhancing Factual Consistency of Abstractive Summarization
Welsh Wizard Gareth Bale opened the scoring in Sunday's stunning win. Real Madrid put nine goals past Granada to keep pressure on Barcelona. Bale treated himself to a BBQ in the Spanish sun following victory. Cristiano Ronaldo scored five goals in the sensational team performance.
Enhancing Factual Consistency of Abstractive Summarization
Gareth Bale scored twice in Real Madrid's 9-1 victory against Granada. The Real Madrid ace scored five goals against Granada on Sunday. The Welsh wizard was in impressive form for the international break.
Enhancing Factual Consistency of Abstractive Summarization
UNILM Gareth Bale scored twice as Real Madrid beat Granada 9-1. The Welsh winger scored for Wales away on international duty. Bale Tweeted:'Unbelievable result this afternoon, great team performance. Now for a BBQ' Real Madrid bounced back from El Clasico defeat by Barcelona with a thumping win. Cristiano Ronaldo scored five goals against an eight-minute hat-trick. 
Enhancing Factual Consistency of Abstractive Summarization
Former Huddersfield centre Josh Griffin scored a try and kicked three goals as Salford secured back-to-back Super League wins with an 18-12 win at John Smith's Stadium. Ben Jones-Bishop scored tries in the first half but Jack Hughes gave the Giants hope. Carl Forster's try and Griffin's penalty made the game safe, though.
Enhancing Factual Consistency of Abstractive Summarization
Huddersfield centre Josh Griffin scored a try and kicked three goals. Griffin scored tries in the first half for the Giants. The game was twice delayed early on when Kevin Locke for Huddersfield's Brett Ferres were carried off injured. Rickie Fowler has earned extra brownie points with his growing legion of fans after calling out an online hater who accused his girlfriend of being a 'gold digger.' The golfer's girlfriend, bikini model Alexis Randock, had posted a photo on her Instagram account of her and sister Nicole on the beach last week... an abusive commenter known as 'fatalsplash' who was quick to judge and accused Alexis of being a 'gold digger.' ... Fowler told the hater to 'get your facts straight' ...
Enhancing Factual Consistency of Abstractive Summarization
Rickie Fowler responded to online troll who abused his girlfriend. Bikini model Alexis Randock had posted a photo on her Instagram account. Randock was pictured on the beach alongside her sister Nicole. A troll called 'fatalsplash' then accused Alexis of being a 'gold digger' Fowler told the hater to 'get your facts straight' following comment.
Enhancing Factual Consistency of Abstractive Summarization
Rickie Fowler accused Alexis of being a 'gold digger' who did n't have to work. The golfer was criticised by an online troll for not working due to her relationship with Fowler. Fowler has earned extra brownie points with his legion of fans.
Enhancing Factual Consistency of Abstractive Summarization
Rickie Fowler 's girlfriend, bikini model Alexis Randock, posted a photo on her Instagram account of her and sister Nicole on the beach last week. An abusive commenter called 'fatalsplash' accused Alexis of being a 'gold digger'
Enhancing Factual Consistency of Abstractive Summarization
FASUM Rickie Fowler has earned extra brownie points with his growing legion of fans. The golfer's girlfriend, bikini model Alexis Randock, had posted a photo on her Instagram account. The photo was shared with her 25,000 followers, which included an abusive commenter known as 'fatalsplash' who was quick to judge and accused Alexis of being a 'gold digger' SEQ2SEQ Golfer's girlfriend posted a picture of her and sister Nicole Randock on Instagram. The picture shared with her 25,000 followers wasshared by her sister on the beach last week. The golfer was quick to judge and accused Alexis of being a 'gold digger' troll. He responded by posting the caption 'fatalsplash' accusing her of being 'a digger who didn't work for it'
Enhancing Factual Consistency of Abstractive Summarization
The VII Summit of the Americas was supposed to be all about the symbolic handshake between the United States and Cuba... Heads of state from 35 countries in the Western Hemisphere have met every three years...Obama and Castro would come face-to-face... Venezuelan President Nicolas Maduro recently accused the United States of trying to topple his government... Reference U.S., Venezuelan relations threaten to overshadow Obama, Castro meeting. Venezuelan President says United States moved to oust him; he has the support of the Cuban foreign minister.
Enhancing Factual Consistency of Abstractive Summarization
Heads of state from 35 countries have met every three years to discuss economic, social or political issues since the summit in 1994. Venezuela's President Raul Castro has been criticized for human rights violations. The U.S. government says the summit of the Americas is a "triangle of tension." UNILM Summit of the Americas was supposed to be all about the symbolic handshake between the U.S. And Cuba. The tide changed when President Obama and Castro announced that more than five decades of Cold War rivalry was ending. Venezuelan President Nicolas Maduro recently accused the U.S. Of trying to topple his government...
Enhancing Factual Consistency of Abstractive Summarization
The summit of the Americas was supposed to be all about the symbolic handshake between the u.s. and Cuba. Diplomats from both countries began negotiations to establish embassies in Havana and Washington. President Nicolas Maduro recently accused the United States of trying to topple his government.
Enhancing Factual Consistency of Abstractive Summarization
The VII summit of the Americas was supposed to be all about the symbolic handshake between the United States and Cuba. Relations between the u.s. and Cuba have been the wrench in diplomatic machinery, with some Latin American leaders threatening not to attend. President Barack Obama and Castro face-to-face embassies in Havana and Washington. 
Enhancing Factual Consistency of Abstractive Summarization
The VII Summit of the Americas was supposed to be all about the symbolic handshake between the United States and Cuba. But insert Venezuela into the mix and Panama City, Panama, quickly turns into a "triangle of tension."... Cuba has historically been the wrench in the diplomatic machinery, with some Latin American leaders threatening not to attend the Summit of the Americas if the United States and Canada didn't agree to invite President Raul Castro... The much anticipated handshake between Obama and Castro would steal all the headlines if it wasn't for Cuba's strongest ally, Venezuela. Venezuelan President Nicolas Maduro recently accused the United States of trying to topple his government and banned former President George Bush...
Enhancing Factual Consistency of Abstractive Summarization
Heads of state from 35 countries have met every three years to discuss economic, social or political issues since the summit in 1994. Venezuela's President Raul Castro has been criticized for human rights violations. The u.s. government says the summit of the Americas is a "triangle of tension."
Enhancing Factual Consistency of Abstractive Summarization
Heads of state from 35 countries have met every three years to discuss economic, social or political issues since the summit in 1994. Venezuela's President Nicolas Maduro has been criticized for human rights violations. The u.s. government says the summit of the Americas is a "triangle of tension."
Enhancing Factual Consistency of Abstractive Summarization
She's one of the hottest and most successful Latinas in Hollywood, but now Sofia Vergara is playing defense in a legal battle initiated by her ex-fiance: He wants to keep the two frozen embryos from their relationship, both female. The 42-year-old actress and star of the hit TV sitcom "Modern Family" split from businessman Nick Loeb in May 2014. Loeb is suing the Colombian-born actress in Los Angeles to prevent Vergara from destroying their two embryos conceived through in vitro fertilization in November 2013, according to published reports by New York Daily News and In Touch magazine...
Enhancing Factual Consistency of Abstractive Summarization
Sofia Vergara wants to keep the frozen embryos from their relationship, both female. He is suing the actress in Los Angeles to prevent Vergara from their embryos. The actress and star of the "Modern Family" split from Nick Loeb in May 2014.
Enhancing Factual Consistency of Abstractive Summarization
Nick Loeb wants to keep the frozen embryos from their relationship, both female. He is suing the actress in Los Angeles to prevent Vergara from their embryos. The actress and star of the"Modern Family"split from Businessman Nick Loeb in May 2014.
Enhancing Factual Consistency of Abstractive Summarization
Volvo says it will begin exporting vehicles made in a factory in southwest China to the United States next month, the first time Chinese-built passenger cars will roll into American showrooms. Parent company Geely Automobile, which bought Volvo in 2010, is seeking to prove that a Chinese company can manage a global auto brand...
Enhancing Factual Consistency of Abstractive Summarization
Volvo is seeking to prove that a Chinese company can manage a global auto brand. The car will be one of four models produced in a manufacturing plant in Chengdu. China is largest market for car sales globally in 2009.
Enhancing Factual Consistency of Abstractive Summarization
Geely Automobile is seeking to prove that a Chinese company can manage a global auto brand. The car will be one of four models produced in a manufacturing plant in Chengdu. China is largest market for car sales globally in 2009.  ... Scott family attorney Chris Stewart said the man with Scott was a co-worker and friend... Slager has been fired and faces life in prison or the death penalty if convicted on a murder charge. Video shot by a witness shows the officer shooting Scott in the back as he runs away. Slager had told investigators he feared for his safety after he and Scott tussled over his Taser. His lawyer, Andy Savage, said Friday he "has not received the cooperation from law enforcement that the media has." ...
Enhancing Factual Consistency of Abstractive Summarization
"This is a heartbreaking tragedy for everyone in our community," Charleston mayor says. Scott family attorney says the man with Scott was a co-worker and friend. Michael Slager has been fired and faces life in prison or the death penalty if convicted on a murder charge. Scott's lawyer says he "has not received the cooperation from law enforcement that the media has Corrected by FC "This is a heartbreaking tragedy for everyone in our community," Charleston mayor says. Scott family attorney says the man with Scott was a co-worker and friend. Michael Slager has been fired and faces life in prison or the death penalty if convicted on a murder charge. Slager's lawyer says he "has not received the cooperation from law enforcement that the media has Article 5
Enhancing Factual Consistency of Abstractive Summarization
Rangers boss Stuart McCall has revealed he was warned about taking over at Ibrox by Walter Smith -but insists he has made the right decision. The former Light Blues midfielder steered his new side to their third victory on the bounce with Sunday's 2-1 win over Hearts. It was a crucial triumph that moves Gers back into second place in the Scottish Championship and looks to have restored vital momentum ahead of the end-of-season play-offs. Stuart McCall says he was warned not to take the Rangers job but is pleased that he took the risk in the end.
Enhancing Factual Consistency of Abstractive Summarization
Stuart McCall guided Rangers to a 2-1 win over Hearts on Sunday. Kenny Miller and Haris Vuckic scored the goals for the Gers at Ibrox. The win moved Rangers back into second place in the Scottish Championship. The Gers boss says he was warned not to take over by Walter Smith. But he says he is pleased that he took the risk in the end.
Enhancing Factual Consistency of Abstractive Summarization
Stuart McCall guided Rangers to a 2-1 win over Hearts on Sunday. Kenny Miller and Haris Vuckic scored the goals for the Gers at Ibrox. The win moved Gers back into second place in the Scottish Championship. The Rangers boss says he was warned not to take over by Walter Smith. But he says he is pleased that he took the risk in the end.
Enhancing Factual Consistency of Abstractive Summarization
Wasps will appeal against the three-week ban issued to Nathan Hughes, but their protest will not enable the back row to play in Sunday's Champions Cup quarter-final against Toulon. The club have said in a statement they will appeal on the grounds the incident that left Northampton wing George North unconscious and resulted in a red card for Hughes was an accident and no foul play had occurred... Wasps are set to appeal the three-match ban handed to No 8 Nathan Hughes for knocking out George North... Hughes was given the suspension -to include three meaningful fixtures -after being found guilty of striking North in the 52-30 Aviva Premiership defeat at Franklin's Gardens on March 27...
Enhancing Factual Consistency of Abstractive Summarization
Wasps will appeal the three-match ban handed to No 8 Nathan Hughes. The Northampton winger was found guilty of striking George North. The incident occurred during Northampton's 52-30 defeat at Franklin's Gardens. North suffered a third confirmed concussion in just four months.
Enhancing Factual Consistency of Abstractive Summarization
Wasps will appeal the three-match ban handed to No 8 Nathan Hughes. The Wasps winger was found guilty of striking George North. The incident occurred during Wasps's 52-30 defeat at Franklin's Gardens. North suffered a third confirmed concussion in just four months.  Table 11 
Factual Error Correction for Abstractive Summarization Models
Self-supervised methods have achieved success in a wide range of NLP tasks, and automatic summarization is no exception (Liu and Lapata, 2019;Lewis et al., 2019;Zhang et al., 2019a;Shi et al., 2019;Fabbri et al., 2019). These state-of-theart abstractive summarization models typically finetune pre-trained transformer-based models on a summarization dataset (Vaswani et al., 2017). Despite significant improvements over previous methods in terms of automatic evaluation scores such as ROUGE (Lin, 2004), ensuring factual consistency of the generated summary with respect to the source remains challenging. For example, 1 Our data and code is available at https://github.com/mcao610/Factual-Error-Correction Source: Jerusalem (CNN)The flame of remembrance burns in Jerusalem, and a song of memory haunts Valerie Braham as it never has before. This year, Israel's Memorial Day commemoration is for bereaved family members such as Braham. "Now I truly understand everyone who has lost a loved one," Braham said. (...) Original: France's memorial day commemoration is for bereaved family members as braham. (inconsistent) After Correction: Israel's memorial day commemoration is for bereaved family members as braham. (consistent) Table 1: An example of an inconsistent systemgenerated summary and the output summary from our correction model. In this case, "France" is successfully corrected as "Israel". Cao et al. (2018) claims that about 30% of summaries generated by abstractive models contain factual errors, which greatly limits their practicality.
Factual Error Correction for Abstractive Summarization Models
Different approaches have been proposed to detect or ensure the factual consistency of generated summaries, including using fact extraction or applying attention on fact triples (Cao et al., 2018;Zhang et al., 2019b;Goodrich et al., 2019), applying natural language inference or question answering models for consistency checking (Falke et al., 2019;Li et al., 2018;Wang et al., 2020) and training the model on artificial datasets (Kryściński et al., 2019). Most of these approaches either require a high-quality fact extraction model or they only focus on factual consistency evaluation. Improving factuality correction by editing inconsistent parts in generated summaries is a direction that has not been explored much.
Factual Error Correction for Abstractive Summarization Models
Previous work on factual consistency in abstractive summarization can be divided into two categories: abstractive summarization models tailored towards factual consistency (Cao et al., 2018;Zhang et al., 2019b;Li et al., 2018), and evaluation models for factual consistency in abstractive summarization (Goodrich et al., 2019;Falke et al., 2019;Kryściński et al., 2019;Wang et al., 2020). Cao et al. (2018) proposed a dual attention module in an abstractive summarizer that attends to both the source document and to relation triples extracted from the document. Zhang et al. (2019b) propose to improve their abstractive summarization model by optimizing fact scores defined in radiology reports with reinforcement learning methods. Li et al. (2018) jointly train their model's encoder on summarization and NLI tasks. Guo et al. (2018) train an abstractive summarization system with the auxiliary tasks of question and entailment generation and show that their generated summaries are less likely to produce extraneous facts. Kumar and Cheung (2019) show that neural abstractive summarizers often assign higher posterior likelihood to perturbed contrastive summaries that are inconsistent with the source text than to human-written gold-standard ones. Concurrently to our work, Zhu et al. (2020) recently proposed a fact-aware summarization model that uses a knowledge graph. They use a pre-trained corrector module to modify generated summaries. Concurrent to our work, Dong et al. (2020) proposes factual correction models that leverages knowledge learned from question answering models via span selection. Their models employ single or multimasking strategies to either iteratively or autoregressively replace entities.
Factual Error Correction for Abstractive Summarization Models
Given a source text d and the reference summary s, we corrupt the reference summary into an inconsistent summary s ′ with a randomly sampled corruption rule (described below) with probability α; otherwise, we keep s ′ = s with probability 1 − α. We set α = 0.3 to match the factuality error rate in real abstract summaries based on a recent study (Cao et al., 2018). The training data consists of triplets (s ′ , s, d).
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
Large pre-trained Transformers have yielded remarkable performance on abstractive summarization (Liu and Lapata, 2019;Lewis et al., 2020;Zhang et al., 2020a) with impeccable fluency, yet their summaries often contain factually inconsistent content (Maynez et al., 2020;Zhang et al., 2020b;Goyal and Durrett, 2020), even for stateof-the-art models. Three types of remedies have been proposed: running a separately learned error correction component (Dong et al., 2020), removing noisy training samples (Nan et al., 2021;Goyal and Durrett, 2021), and modifying the Transformer architecture (Huang et al., 2020;Zhu et al., 2021). Yet they either rely on heuristically created data for error handling, falling short of generalization, or require learning a large number of new parameters, and summary informativeness is often sacrificed. XSum Article: The Fermanagh MLA Phil Flanagan tweeted after Tom Elliott appeared on a BBC radio programme in May 2014. . . . "I wonder if he will reveal how many people he harassed and shot as a member of the UDR.". . .
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
Contrastive learning (our method): A Sinn Féin MLA has been ordered to apologise and pay compensation to a former member of the Ulster Defence Regiment (UDR).
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
Cross-entropy: A Sinn Féin MLA has agreed to pay compensation to a former Ulster Unionist Party (UDR) MP after he tweeted that he had harassed and shot people as a member of the party.
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
Entailment reranking: A Sinn Féin MLA has agreed to pay compensation to a former Ulster Unionist Party (UDR) councillor for a tweet he sent about him.
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
Unlikelihood: An MLA has been ordered to apologise and pay compensation to a former loyalist MP for a remark he made about him while serving in the Ministry of Defence. Figure 1: Sample article and system summaries by different methods. Our contrastive learning model trained on low confidence system outputs correctly generates the full name. Comparisons using cross-entropy loss, beam reranking by entailment scores (Kryscinski et al., 2020), and unlikelihood objective (Welleck et al., 2020) over negative samples all produce unfaithful content.
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
Our goal is to train abstractive summarization systems that generate both faithful and informative summaries in an end-to-end fashion. We observe that, while the commonly used maximum likelihood training optimizes over references, there is no guarantee for the model to distinguish references from incorrect generations (Holtzman et al., 2020;Welleck et al., 2020). Therefore, potential solutions reside in designing new learning objectives that can effectively inform preferences of factual summaries over incorrect ones.
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
Concretely, we hypothesize that including factually inconsistent summaries (i.e., negative samples) for training, in addition to references (i.e., positive samples), let models become better at differentiating these two types of summaries. Although using negative samples has been effective at text representation learning, e.g., word2vec (Mikolov et al., 2013) and BERT (Devlin et al., 2019), there exist two major challenges for it to succeed in concrete language tasks. First, a suitable training objective is critical to avoid performance degradation (Saunshi et al., 2019). Second, it is nontrivial to construct "natural" samples that mimic the diverse errors made by state-of-the-art systems that vary in words and syntax (Goyal and Durrett, 2021).
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
To address both challenges, we first propose a new framework, CLIFF, that uses contrastive learning for improving faithfulness and factuality of the generated summaries. 1 Contrastive learning (CL) has obtained impressive results on many visual processing tasks, such as image classification (Khosla et al., 2020;Chen et al., 2020) and synthesis (Park et al., 2020;Zhang et al., 2021b). Intuitively, CL improves representation learning by compacting positive samples while contrasting them with negative samples. Here, we design a task-specific CL formulation that teaches a summarizer to expand the margin between factually consistent summaries and their incorrect peers.
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
Moreover, we design four types of strategies with different variants to construct negative samples by editing reference summaries via rewriting entity-/relation-anchored text, and using system generated summaries that may contain unfaithful errors. Importantly, these strategies are inspired by our new annotation study on errors made by state-of-the-art summarizers-models fine-tuned from BART (Lewis et al., 2020) and PEGA-SUS (Zhang et al., 2020a)-on two benchmarks: XSum (Narayan et al., 2018) and CNN/DailyMail (CNN/DM) (Hermann et al., 2015).
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
We fine-tune pre-trained large models with our contrastive learning objective on XSum and CNN/DM. Results based on QuestEval (Scialom et al., 2021), a QA-based factuality metric of high correlation with human judgments, show that our models trained with different types of negative samples uniformly outperform strong comparisons, including using a summarizer with post error correction and reranking beams based on entailment scores to the source. Moreover, compared with unlikelihood training method that penalizes the same negative samples (Welleck et al., 2020), our summaries also obtain consistently better QuestEval scores. Human evaluation further confirms that our models consistently reduce both extrinsic and 1 Our code and annotated data are available at https:// shuyangcao.github.io/projects/cliff_summ. intrinsic errors over baseline across datasets.
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
We design a contrastive learning (CL)-based training objective that drives the summarization model to learn a preference of faithful summaries over summaries with factual errors. It is then used for fine-tuning BART (Lewis et al., 2020) and PEGA-SUS (Zhang et al., 2020a) for training summarization models. Formally, let an article x have a set of reference summaries P (henceforth positive samples) and another set of erroneous summaries N (negative samples). The contrastive learning objective is (Khosla et al., 2020;Gunel et al., 2021):
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
where h i , h j , and h k are representations for summaries y i , y j , and y k . sim(·, ·) calculates the cosine similarity between summary representations. τ is a temperature and is set to 1.0. Importantly, summaries in P and N are included in the same batch during training, so that the model acquires better representations to differentiate correct summaries from those with errors by comparing the two types of samples, thus maximizing the probabilities of the positive samples and minimizing the likelihoods of the corresponding negative samples. The CL objective on the full training set, denoted as L CL , is the sum of losses l x cl over all samples.
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
To effectively employ CL in summarization, we need to address two challenges: (1) how to automatically construct both positive and negative samples, which are critical for CL efficacy (Chen et al., 2020), and (2) how to represent the summaries (i.e., h * ). Below we describe positive sample generation and options for h * , leaving the strategies for negative samples to § 5.
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
Positive Sample Construction (P ). Summarization datasets often contain a single reference for each article. To create multiple positive samples, in our pilot study, we experiment with paraphrasing with synonym substitution (Ren et al., 2019), randomly replacing words based on the prediction of masked language models (Kobayashi, 2018), and back-translation (Mallinson et al., 2017). We find back-translation to be best at preserving meaning and offering language variation, and thus use NL-PAug 2 to translate each reference to German and back to English. Together with the reference, the best translation is kept and added to P , if no new named entity is introduced.
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
Summary Representation (h * ). We use the outputs of the decoder's last layer, and investigate three options that average over all tokens, named entity tokens, and the last token of the decoded summary. Entities and other parsing results are obtained by spaCy (Honnibal et al., 2020). We further consider adding a multi-layer perceptron (MLP) with one hidden layer to calculate the final h * .
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
The final training objective combines the typical cross-entropy loss L CE and our contrastive learning objective: L = L CE + λL CL , where λ is a scalar and set to 1.0 for all experiments. 
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
We first describe annotating unfaithfulness errors by state-of-the-arts, i.e., models fine-tuned from BART and PEGASUS on XSum and CNN/DM. We then probe into the model generation behavior that is indicative of errors, which guides the design of negative sample construction strategies. 600 (150 × 2 × 2) summaries are annotated to demonstrate how often do the models "hallucinate", i.e., generating content not grounded by the source. To characterize errors, we annotate text spans in summaries with (i) intrinsic errors caused by misconstructing phrases or clauses from the source; and (ii) extrinsic errors which include words not in the source that are either unverifiable or cannot be verified by Wikipedia. Content not covered by the article but can be validated by Wikipedia is annotated as world knowledge, and the models' behavior pattern when generating them differs from when they generate errors.
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
Two fluent English speakers with extensive experience in summary evaluation and error labeling are hired. For each sample, they are shown the article and two system summaries, and instructed to annotate text spans with the aforementioned errors and world knowledge. After labeling every 50 samples, the annotators discuss and resolve any disagreement. The Fleiss's Kappas on XSum and CNN/DM are 0.35 and 0.45.
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
Error statistics are displayed in Fig. 2. Extrinsic errors dominate both datasets, especially on XSum. 58.7% of summaries by BART (and 44.0% by PE-GASUS) contain at least one extrinsic error. Noticeably, PEGASUS is a newer model pre-trained with a larger amount of data, thus contains less errors than BART and other older models studied for error annotations by Maynez et al. (2020). This observation also highlights the usage of our anno- tations for future development and evaluation of summarization models.
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
Low confidence generation is indicative of extrinsic errors. Inspired by recent work that studies model prediction confidence (Liu et al., 2021), we examine generation probabilities for tokens of different part-of-speech (POS) tags. Fig. 3 shows salient results on the generation probabilities of the first token of a proper noun or a number (with additional analysis provided in Appendix A). As observed, model confidence tends to be lower for the first tokens of proper nouns and numbers if they are part of spans with extrinsic errors. Also note that world knowledge, which cannot be inferred from the source either, often has higher generation probability than extrinsic errors. Take this snippet generated by a fine-tuned BART as an example: "Manchester United captain Wayne Rooney's testimonial game against Manchester City. . .". "Manchester City" is an extrinsic error and "Wayne" is produced as world knowledge. The model assigns a low probability of 0.10 to the first token of "Manchester City" and a high probability of 0.92 to token "Wayne". This implies that model confidence can be a useful indicator for negative sample collection.
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
Here we describe four strategies for constructing negative samples that modify the references ( § 5.1-5.3) or use system generated summaries (5.4).
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
Entity swap imitates intrinsic errors, as over 55% of intrinsic errors in our annotations are found to contain named entities. We construct negative samples by swapping named entities in the references with other randomly selected entities of the same entity type in the source (SWAPENT). One sam-REFERENCE: A "rare" short-eared owl found emaciated in Flintshire is now recuperating well, the RSPCA have said.
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
SWAPENT: Flintshire → Bettisfield ⇒ A "rare" short-eared owl found emaciated in Bettisfield is now recuperating well, the RSPCA have said.
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
MASKENT: A "rare" short-eared owl found emaciated in [MASK] is now recuperating well, the RSPCA have said. ⇒ A "rare" short-eared owl found emaciated in a field in South Yorkshire is now recuperating well, the RSPCA have said.
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
A "rare" short-eared owl found [MASK] in [MASK] is now recuperating well, the RSPCA have said. ⇒ A "rare" short-eared owl found dead in London is now recuperating well, the RSPCA have said.
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
REGENENT: A "rare" short-eared owl found emaciated in ⇒ A "rare" short-eared owl found emaciated in Nottinghamshire is now at a wildlife centre to recover. REGENREL: A "rare" short-eared owl found ⇒ A "rare" short-eared owl found in the grounds of a former coal mine is being cared for by the RSPCA in Somerset.
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
SYSLOWCON: An injured golden owl found in a former coal mine in Lancashire is being cared for by the RSPCA.  Table 1. SWAPENT has the advantage of not depending on any trained model. Yet it only introduces intrinsic errors and lacks the coverage for extrinsic errors, which is addressed by the following generationbased methods.
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
To simulate extrinsic errors, we leverage large unconditional language models' capability of converting a sequence with masked tokens into a fluent and appropriate sequence. Specifically, we replace each named entity in a reference with a [MASK] token and encode it with BART (without any fine-tuning). BART then fills this partially masked summary with newly generated entities (MASKENT). BART is chosen since it can fill [MASK] with varying number of tokens. For each entity in the reference, we sample three summaries and only retain the ones containing at least one entity that is absent from both the source and the reference.
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
Up to now, the two introduced strategies both focus on incorrect named entities. To cover more diverse extrinsic and intrinsic errors (Goyal and Durrett, 2020), we extend MASKENT to contain relations (MASKREL). We first obtain dependency relations using Stanza (Qi et al., 2020), with each relation denoted as <gov, rel, dep>. To incorporate more context, we consider noun phrase spans enclosing the token of gov or dep if it is a content word and the noun phrase contains a named entity. Similar to MASKENT, three negative samples are generated by BART based on the input with both gov and dep spans masked in the reference. Only the samples that introduce any new dependency relation that is not contained in the source nor the reference are kept. Specifically, we consider a match of a dependency relation as the same form or synonyms of its gov and and dep is found in the source or the reference with the same relation.
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
Both MASKENT and MASKREL can create more extrinsic errors compared to other strategies introduced in this section, since negative samples are generated without being grounded on the source articles. However, their constructed negative samples may contained drifted topics that can be easily detected by a summarization model, resulting with less efficient training signals.
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
To ground negative sample generation with the article, we further design a regeneration strategy based on conditional generation. For each named entity in the reference, we treat the text before it as a prompt. A summarizer, e.g., fine-tuned from BART or PEGASUS, first reads in the source using the encoder, then receives the prompt as the first part of the decoder output, and finally decodes the rest of the content based on nucleus sampling (Holtzman et al., 2020) with a cumulative probability threshold of 0.7. The prompt and the regenerated text comprise the final negative sample. This method is denoted as REGENENT.
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
We also extend entities to relations with expanded governor and dependent spans (REGENREL). Here, we consider a prompt as the text before the gov or dep span, whichever occurs first. For both REGENENT and REGENREL, three negative samples are generated for each prompt, and a sample is kept if it introduces any new entity (for REGENENT) or dependency relation (for REGENREL) with regard to the source and the reference.
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
Negative samples generated by both methods are more relevant to the article than the mask-and-fill strategy, yet they may still miss certain types of errors and differ from real model outputs, since they are modified from the reference summaries.
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
Motivated by the model confidence analysis in § 4, we explore using system generated summaries as negative samples. We first run fine-tuned BART or PEGASUS on the same training set to decode summaries. For each summary, we check the model confidence on the first token of each proper noun and number span. If the probability is below a threshold, we keep it as a negative sample (SYSLOWCON). The threshold is tuned by maximizing F1 based on our error annotations.
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
We consider all beams at the last decoding step as candidates. We use beam sizes of 6 and 4 for XSum and CNN/DM. Statistics of negative samples constructed by different strategies are in Appendix B.
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
Evaluation Metrics. QuestEval (Scialom et al., 2021) is used as the main metric to evaluate summaries' factual consistency. Given an article and a summary, QuestEval first generates natural language questions for entities and nouns from both. A QA model then consumes the article to answer questions derived from the summary, producing a score. Another score is obtained from a QA model addressing article-based questions after reading the summary. The final QuestEval score is the harmonic mean of the two. We use the version with learned weights for questions, which has shown high correlation with human judged consistency and relevance.
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
We further use FactCC (Kryscinski et al., 2020), trained based on their negative sample construction method, to measure if the summary can be entailed by the source. We also report ROUGE-L (Lin, 2004). Both FactCC and ROUGE-L reasonably correlate with summary factuality as judged by human (Pagnoni et al., 2021).
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
Based on our error annotations, we report the correlations between each metric and the error ratepercentage of tokens being part of an error span, and the raw number of errors (Table 2). QuestEval correlates better on both aspects than other metrics.
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
Comparisons. In addition to the models finetuned with cross-entropy loss (CRSENTROPY), we consider reranking beams based on FactCC
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
XSum CNN/DM % of Err # of Err % of Err # of Err  (Li et al., 2020). Given a negative sample y , the loss is defined as − |y | t=1 log(1 − p(y t |y 1:t−1 , x)), where p(y t |y 1:t−1 , x) is the output probability at the t-th step. We combine the unlikelihood training objective with cross-entropy loss with equal weights for fine-tuning.
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
Lastly, we compare our negative sample strategies with negative samples constructed for training the FactCC scorer, denoted as FCSAMPLE. For CL only, we compare with using other samples in the same batch as negative samples (BATCH), a common practice for CL-based representation learning (Gao et al., 2021;Zhang et al., 2021a).
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
We report results by models fine-tuned from BART and PEGASUS with different objectives and negative samples on XSum and CNN/DM in Tables 3  and 4. CLIFF models use a summary representation of averaging over all tokens with MLP projection, with other variants discussed in § 7.3. Unless explicitly stated, comparison models are fine-tuned from the same large model used by CLIFF.
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
First, comparing with other factuality improvement models (top of the tables), almost all CLIFF models trained with different negative samples uni-  Table 3: Results of models fine-tuned from BART on XSum and CNN/DM. QEval: QuestEval; FC: FactCC; R-L: ROUGE-L. The best result per metric per dataset is bolded. For models of unlikelihood training and CLIFF that use the same negative samples, the better of the two is highlighted with green. * : our model is significantly better than CRSENTROPY (approximation randomization test, p < 0.005).
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
formly produce higher QuestEval scores across datasets with both large models, with the improvements more pronounced on XSum. Importantly, ROUGE scores for CLIFF models are comparable or better than baselines trained with crossentropy, e.g., on CNN/DM as in Table 3. A similar trend is observed with the FactCC metric, especially when using PEGASUS as the base model ( Table 4). Note that ENTAILRANK tends to yield significantly higher FactCC scores, though it obtains lower QuestEval scores than the cross-entropy baseline. Human inspection finds that ENTAIL-RANK can pick up beams with peculiar words of high FactCC scores, without improving factuality. Moreover, other comparisons based on post COR-RECTION and model engineering (FASUM) only offer incremental gains. The sample selection-based method, SUBSETFT, sacrifices ROUGE scores significantly. Overall, CLIFF demonstrates stronger generalizability.
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
Second, CLIFF is more effective and robust than  Table 4: Results of models fine-tuned from PEGASUS on XSum and CNN/DM. We report results on 5, 000 randomly selected samples on CNN/DM, due to long running time of QuestEval. For models of unlikelihood training and CLIFF that use the same negative samples, the better of the two is highlighted with green. * : our model is significantly better than CRSENTROPY (approximation randomization test, p < 0.005).
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
unlikelihood training with the same negative samples. According to Table 3, using 7 negative sample construction strategies on two datasets, CLIFF obtains higher QuestEval scores than unlikelihood training in 12 out of the 14 comparisons. Using PEGASUS, CLIFF also outperforms in 11 setups as listed in Table 4. Similar trends are found on FactCC and ROUGE-L. Another noteworthy piece is that CLIFF's improvements over the crossentropy baseline are more consistent, whereas unlikelihood training occasionally hurts factuality or ROUGE scores significantly. We believe the key advantage of CLIFF resides in its measure of representation similarities between positive and negative samples in the same batch, allowing models to better differentiate between correct and erroneous summaries.
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
Finally, among all variants, CLIFF trained with low confidence summaries as negative samples obtains the best QuestEval scores on the more abstractive dataset. As seen in Table 3, using low confidence summaries also improves FactCC scores on both datasets, and enhances ROUGE-L on the more  extractive dataset CNN/DM. This indicates that system generated summaries contribute more diverse errors made by existing models organically, which are particularly suitable for our CL framework. As we use summaries generated by the same model for CLIFF training, one future direction is to use outputs by different models. For our mask-and-fill and source-conditioned regeneration strategies, we find that relation-anchored construction often beats their entity-anchored counterparts. This calls for efforts that steer the entity-driven methods to a more relation-focused direction.
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
Combining Strategies. We further show results by fine-tuning BARTs using samples based on combined negative sample construction strategies in Table 5. As can be seen, combining SYSLOWCON and other strategies yields better QuestEval scores than models trained with negative samples by any single strategy, except for MASKENT and REGE-NENT on XSum. This signifies the importance of covering diverse types of errors in negative samples.
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
Pairwise Comparison with Cross-entropy. We recruit the two human annotators for our summary error study, as well as another experienced annotator, to evaluate summary informativeness and factual consistency. For each article, the judges are shown summaries generated by the CRSENTROPY model and four other systems. They then rate each system summary against the CRSENTROPY summary. All four summaries generated by different factuality-improved models are shown in random order without system names shown, ensuring the fair comparison among them. We randomly pick 100 articles from each dataset used in our error analysis study in § 4, and evaluate summaries generated by ENTAILRANK, unlikelihood training (ULL) with negative samples con-   Table 6: Percentages of summaries that are better than, tied with, or worse than CRSENTROPY, in informativeness (Inform.) and factual consistency (Factual.) The Krippendorff's αs are 0.33 and 0.62 for the two aspects on XSum, and 0.34 and 0.89 on CNN/DM. Our CL method using low confidence summaries is more frequently rated as better for informativeness and factuality on the more abstractive dataset XSum.
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
Extrinsic 0   structed by MASKENT, and CLIFF models trained with BATCH and SYSLOWCON negative samples. All are fine-tuned from BART. Detailed evaluation guidelines are in Appendix D. Table 6 shows that on the more abstractive XSum data CL trained with low confidence samples are more frequently rated as being more informative and more factual than CRSENTROPY summaries. This echos our automatic evaluations with QuestEval in § 7.1. On CNN/DM, all models trained with negative samples produce summaries with better informativeness and faithfulness. In contrast, ENTAILRANK summaries are less distinguishable from outputs by CRSENTROPY on both datasets, as more ties are found. We show sample outputs in extrinsic errors as done in § 4. Fig. 4 shows that CL is more effective at reducing extrinsic errors than unlikelihood training can on both datasets. We also observe slight decreases of world knowledge in the summaries (figure attached in Appendix D).
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
Error Correction Operations. Finally, with reference to CRSENTROPY summaries, human judges are instructed to label each system summary as whether it corrects any error by CRSENTROPY using deletion of the incorrect content, substitution with factual information, or both. As seen in Fig. 5, CL-based models restore factually consistent information, e.g., by replacing erroneous names and numbers with correct ones, more frequently than unlikelihood training or entailment reranking.
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
Sample representation is critical for CL to be effective. Here we investigate summary representation variants as discussed in § 3. There are two major considerations: (1) Should we consider all tokens in a summary or only representative ones (e.g., entities or last token)? (2) Should additional transformation, i.e., an MLP, be used?
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
Experiments on XSum using three negative sample construction strategies demonstrate that averaging the decoder outputs of all tokens and adding an MLP projection yield the best overall performance, as shown in Table 7. The implications are at least two-fold. First, even for entity-or relationtriggered sample modifications, using more global context helps with CL training. Second, additional transformation can help avoid model degeneration. For instance, more nonsensical and repetitive content is produced by variants without MLP.  
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
We present CLIFF, a contrastive learning-based framework to promote faithfulness and factuality of abstractive summaries. CLIFF uses both references and summaries that are factually inconsistent with the articles to train systems to be better at discriminating errors from factual and salient content. We further study strategies that automatically create erroneous summaries by editing from references or leveraging systems outputs, inspired by our new summary error analysis on state-of-theart models. Both automatic evaluation and human ratings show that CLIFF achieves consistent improvements over competitive comparison methods, and is generalizable across datasets with systems fine-tuned from different large models. 
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
We hire two fluent English speakers to annotate summary errors on XSum and CNN/DailyMail (CNN/DM). They annotate a common batch of 100 summaries generated by summarizers fine-tuned from BART and PEGASUS, with 50 articles in each batch. The two annotators are shown 50 HTML pages in a batch, each of which contains an article and two summaries generated by the two models. The detailed annotation guideline is given in Fig. 9. For our analysis on token generation probabilities, we additionally show the distributions of the first token's probability for nouns and verbs in Fig. 6. We also report the distributions of the nonfirst token's probability for proper nouns, numbers, nouns, and verbs in Fig. 7. As can be seen, tokens within extrinsic and intrinsic errors have high generation probabilities when they are non-first tokens.
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
Summarization Datasets. We follow the official data splits for the two datasets, with the number of samples in each split listed in Table 8.   Table 9. SYSLOWCON constructs the least negative samples in total, while it achieves the best results as reported in our main paper ( § 7.1), indicating that its negative samples are more effective for training.  
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
In § 7.2, we demonstrate the percentages of samples containing intrinsic errors and extrinsic errors for each model evaluated by human judges. Here, we report the percentages of samples containing world knowledge in Fig. 8. On XSum, all models produce less world knowledge compared to the model trained with cross-entropy loss, while gen-  
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
We include more sample outputs in Fig. 11.
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
In this study, you will first read article-summary pairs and then identify three types of text spans in the summaries. These spans include content that is contradicted by or cannot be implied from the article. The description for each type is described below:
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
• Intrinsic errors: Text spans that misconstruct phrases or clauses from the article.
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
• Extrinsic errors: Text spans that include words that are not in the article and are not verifiable or cannot be verified by Wikipedia.
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
• world knowledge: Text spans that contain information that is not covered by the article but can be validated by Wikipedia.
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
When selecting spans, you should always make sure the spans are complete words.
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
In practice, you should follow the these steps carefully: (1) read the article and summaries carefully;
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
(2) figure out if there is content contradicted by or not presented in the article; (3) label the span as an intrinsic error if it misconstructs phrases or clauses from the article; (4) if the span does not belong to intrinsic errors, search within Wikipedia and determine whether the content in the span can be verified; (5) label it as world knowledge if the it can be verified by Wikipedia, otherwise label it as an extrinsic error.
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
Example annotations 1 Article: Isis Academy in Oxford said it had rebranded as "Iffley Academy" to protect its "reputation, integrity and image". The name 'Isis' was originally chosen as the school is near to the section of the River Thames of the same name. Formerly Iffley Mead School, it became Isis Academy in 2013. A statement issued by the school said it had changed name following "the unforeseen rise of ISIS (also known as ISIL and the Islamic State) and related global media coverage of the activities of the group". "Our priority is to remove the detrimental impact which the name 'Isis' had on pupils, their families and our staff." Last year a language school in the city removed Isis from its name for the same reason. The Isis is the name given to the part of the River Thames above Iffley Lock in Oxford. It is also the name of the goddess wife of the god Osiris in Egyptian beliefs. Summary: A school that was named after the Islamic State (IS) militant group has changed its name. Explanation: "was name after" is an intrinsic error contradicted by the article sentence in bold.
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
Example annotations 2 Article: Khalil Dale, 60, was abducted in Quetta in January 2012 and was found dead on a roadside a few months later. He had been beheaded. A note next to his body said he was killed because a ransom had not been paid. Mr Dale was born in York but lived in Dumfries. He spent 30 years working in countries including Somalia, Afghanistan and Iraq. An inquest into his death was held at Chesterfield Coroners Court because he is buried in Derbyshire. The court heard that the Muslim convert, who was formerly known as Kenneth, worked as a humanitarian assistance relief worker. Following his abduction, negotiations were undertaken by the International Committee of the Red Cross with the help of the UK government. His body was found on 29 April 2012. The inquest was told that he died as a result of decapitation. Senior coroner Dr Robert Hunter concluded that Mr Dale was unlawfully killed while providing international humanitarian assistance. Summary: A British aid worker was unlawfully killed by Islamist militants in Pakistan, an inquest has heard. Explanation: "Islamist militant" is an extrinsic error as it can not be found in or inferred from the article. The information is also not verifiable by Wikipedia. "Pakistan" is world knowledge as Quetta in the article is a city in Pakistan according to Wikipedia. Figure 9: Guideline for our summary error annotation ( § 4).
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
In this study, you will evaluate 100 sets of summaries produced by four systems. For each set, its corresponding article and a baseline summary are shown before the four system summaries. The errors in the baseline summary are highlighted. Please first read the article and the baseline summary and then compare each system summary against the baseline summary based on informativeness and factual consistency. In addition, please decide the operations made by the system to achieve better factual consistency. For informativeness and factual consistency, you need to label whether the system summary is better or worse than the baseline summary. You can also label the system summary as tying with the baseline summary. You need to consider two types of operations: deletions and substitutions. Please label the system summary as making deletions, substitutions, or both operations. Examples for the aspects and the operations are as follows.
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
Article: Alexys Brown, also known as Lexi, died at her home in Emmadale Close, Weymouth, on Thursday. An investigation is under way to discover how she became trapped. A post-mortem examination is due to be carried out this week. It was originally hoped the appeal would raise £2,000. Alison Record, who started the Just Giving appeal, said she was "heart broken" over the death. "Everybody by now has heard of the terrible tragedy the Brown family have suffered with the loss of their beautiful and beloved little girl Lexi," the appeal page reads. Many other comments have been posted on the appeal page. Steph Harris said: "Thinking of you all at this devastating time, fly high beautiful princess. Love Steph and family xxx" Lesley Andrews added: "No amount of money will take away the pain, but so much love comes with every penny. Take care. xx" Aster Group, the housing association responsible for managing the home, is assisting with the police investigation. The Health and Safety Executive (HSE) is also investigating. Dorset County Council said it had not installed the disabled lift at the property. Baseline Summary: An appeal to raise 10,000 pounds for the family of a three-year-old girl who died after becoming trapped in a lift has raised more than 20,000 pounds.
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
Informativeness: Whether the summary captures salient content from the input article. Note that incorrect content should be considered as invalid.
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
Win. An appeal to raise money for the family of a three-year-old girl who died after getting stuck in a lift was originally hoped for raising £2,000. The target money of the appeal is a salient information.
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
Tie. An appeal to raise money for the family of a girl who died after getting stuck in a lift has raised more than £20,000. Compared to the baseline, missing incorrect information does not affect the informativeness.
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
Lose. An appeal to raise money for the family of a three-year-old girl has raised more than £20,000. This system summary does not mention the death of the girl, which is a salient content of the article.
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
Factual Consistency: Whether the summary is factually correctly based on the article and knowledge from Wikipedia.
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
Win. An appeal has been set up for the family of an eight-year-old girl who died after becoming trapped in a lift at her Dorset home. This system summary does not generate the incorrect numbers of money.
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
Tie. An appeal to raise 5,000 pounds for the family of a seven-year-old girl who died after becoming trapped in a lift has raised more than 20,000 pounds. This system summary makes similar errors to the baseline.
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
Lose. The family of an eight-year-old girl who died after becoming trapped in a lift at her Dorset home have set a fundraising target of 10,000 pounds. This system summary fabricates an event The family have set a fundraising target, which is more severe than errors of modifiers.
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
Deletion: The incorrect content in the baseline summary is deleted.
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
-An appeal for the family of a three-year-old girl who died after becoming trapped in a lift has raised more than 20,000 pounds. The error "10,000 pounds" is deleted. Substitution: The incorrect content in the baseline summary is replaced with correct one.
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
-An appeal to raise 2,000 pounds for the family of a three-year-old girl who died after becoming trapped in a lift has raised more than 20,000 pounds. The error "10,000 pounds" is substituted with "2,000 pounds", which is the correct information. 
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
State-of-the-art abstractive summarization systems can generate fluent summaries with high automatic evaluation scores in terms of ROUGE (Lin, 2004). However, recent studies have shown that these systems are prone to hallucinate content that is not supported by the source document (Maynez et al., 2020;Kang and Hashimoto, 2020;Durmus et al., 2020;Filippova, 2020;Kryscinski et al., 2020). For instance, Maynez et al. (2020) discovered that 64.1% of the summaries generated by a BERT-based abstractive summarization model on XSUM (Narayan et al., 2018a) contain hallucinations.
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
Previous studies commonly assume that hallucination is an undesirable behavior in abstractive summarization systems. They investigate the 1 https://github.com/mcao516/EntFA
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
Under the proposals, 120,000 additional asylum seekers will be distributed among EU nations, with binding quotas. (...) Mr Juncker told the European Parliament it was "not a time to take fright". (...) He said tackling the crisis was "a matter of humanity and human dignity". "It is true that Europe cannot house all the misery in the world. But we have to put it into perspective." (...) Generation: European Commission President Jean-Claude Juncker has set out his proposals for dealing with the migrant crisis in a speech to MEPs, saying Europe cannot house all the misery in the world. Table 1: Example of factual hallucinations in a BART generated summary on XSUM. Neither the title "European Commission President" nor the first name "Jean-Claude" is mentioned in the document but both are factual.
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
cause of model hallucination (Kang and Hashimoto, 2020;Wang and Sennrich, 2020) and propose methods that reduce the frequency of all hallucinations (Filippova, 2020;Nan et al., 2021;Narayan et al., 2021).
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
Our stance in this paper is that hallucinations are not always undesirable: many factual hallucinations provide additional world knowledge that is important for summary comprehension. Table 1 presents one such example from XSUM: the hallucinated content European Commission President provides additional background information on the role of Mr. Juncker. Factual hallucinations refer to content that is verifiable by world knowledge but not inferable from source text.
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
We thus argue that not all hallucinations should be treated equally; in particular, factual hallucinations may be less deleterious or even potentially beneficial to to be included in a summary, as opposed to non-factual ones. We propose a method to classify entities according to whether they are hallucinations and whether they are factual (if hallucinated). We focus on entities (e.g., persons, locations, dates, cardinal numbers) because they are necessary to express the most salient pieces of in-formation in a summary. Moreover, entity hallucinations are common in generated summaries. As we will show later in our work, about 30% of entities generated by BART (Lewis et al., 2020) on XSUM test set are hallucinated.
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
Our approach is inspired by the observation that many hallucinated entities are generated with low probabilities. This observation suggests that the summarization model's confidence correlates with the factuality statuses of generated entities. In other words, the uncertainty is indicative of the likelihood of whether generated entities are hallucinated and non-factual.
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
We refer to the probability of an entity being in a summary without considering the source document as its prior probability, and its probability given the document as its posterior probability. Our assumption is that if an entity in a generated summary results in a factual error, giving the source should not provide more evidence for it, resulting in a small change in probability between the prior and the posterior. Based on this assumption, we propose to use the prior and posterior probabilities as the key features in a simple classifier that predicts an entity's hallucination status and factuality.
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
Due to the lack of fine-grained hallucination annotation, we create an entity-level hallucination and factuality annotation on the XSUM dataset. We evaluate our detection method on this annotated dataset as well as annotations from Maynez et al. (2020). On both datasets, our approach outperforms five baseline models at identifying nonfactual hallucinations. In addition, our approach has a strong correlation with the factuality scores given by human judges. Besides, we show that our detector, when used as a reward signal in training neural-based summarizers with the off-line RL algorithm, significantly improves the factuality of generated summaries even when the underlying dataset is noisy.
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
Our contributions are the following: (i) We demonstrate that an entity's prior and posterior probabilities can be used to infer whether it is hallucinated and factual. Based on this hypothesis, we propose a novel approach for entity-level hallucination detection and factuality checking. Our approach outperforms five baselines from previous work on two human-annotated datasets, in addition to having a strong correlation with summarylevel factuality scores given by human judges. (ii) We empirically demonstrate that our classifier can provide reliable reward signals for RL algorithms, leading to improved factuality while maintaining the level of abstractiveness in generated summaries. (iii) We create a set of entity-level hallucination annotations.
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
Maynez et al. (2020) conducted a large-scale human evaluation of several neural abstractive summarization systems, and found that hallucinations are common among the outputs of different summarization models.
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
Recently, many methods have been proposed to reduce model hallucination. Kang and Hashimoto (2020) propose a "loss truncation" training algorithm that filters out noisy training samples which may lead a model to hallucinate.  use a verification system to recognize non-factual quantities in summaries and adopt a re-ranking system to reduce the number of hallucinated quantities in the final output summary. Narayan et al. (2021) use entity chains to mitigate the hallucination problem in the generation of abstractive summaries. Nan et al. (2021) show that data filtering and use a summary-worthy entity classification task as an auxiliary training objective can help improve model's entity-level factuality.
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
Filippova (2020) proposed a method for controlling hallucination in data-to-text generation task. They suggest that a conditional language model (CLM) will put more probability mass on a nonhallucinated entity than an unconditional language model (LM). Our work differs in that we focus on both hallucination and factuality. Also, our method works at the entity-level rather than the sentencelevel, and is geared towards text summarization.
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
Another line of work focuses on evaluating the factual consistency of abstractive summarization systems. Kryscinski et al. (2020) train models on an artificially corrupted dataset for factual errors detection. Cao et al. (2020) induce artificial perturbations in text to train a summary error correction system, but find that there is a large gap between such artificial perturbations and the type of hallucinations that are generated by abstractive summarizers. (Goyal and Durrett, 2020) measure factual consistency by checking whether the semantic relationship manifested by individual dependency arcs in the generated summary is supported by the source document. ; Dong et al. (2020);Durmus et al. (2020) measure and improve the factual consistency of summaries by asking and answering questions based on generated summaries and input documents.
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
In this section, we propose a novel detection approach that separates factual from non-factual hallucinations of entities (Section 3.2), and present a factuality-aware training framework for summarization models trained on noisy dataset (Section 3.3).
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
Let (S, R) be a pair of a source document and a reference summary, where S = (s 1 , ..., s M ) is the source document with M tokens, and R = (r 1 , ..., r L ) is the reference summary with L tokens. Let G = (g 1 , ..., g N ) be the model-generated summary with N tokens. For each named entity e k , which we assume to be a span of tokens g i k , ..., g i k +|e k |−1 (|e k | ≥ 1) starting at position i k in G, the task is to determine whether e k is hallucinated, and whether it is factual. We define an entity as hallucinated if it is not directly inferable in its generated context given the input document S. If an entity is hallucinated, we further classify it into two subtypes: factual hallucinations and non-factual hallucinations. Factual hallucinations cannot be directly entailed in their generated context from the source document but can be based on world knowledge (see Table 1). Non-factual hallucinations are entities that are neither inferable from the source nor based on world knowledge.
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
We now define the prior and posterior probabilities of an entity, which we will use to predict its hallucination and factuality statuses.
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
For entity e k , we define its prior probability p prior (e k ) as the probability of its generation by a language model that does not have access to the source text. If e k spans multiple tokens, we compute its probability auto-regressively. Let c k be the context of entity e k in G, excluding the tokens in e k . Then:
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
which we compute using a masked language model P MLM . The posterior probability p pos (e k ) of entity e k is the conditional probability of the entity given the context and the source text:
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
where CMLM is a conditional masked language model. CMLM is an encoder-decoder model that is trained with a masked language model objective on a parallel dataset. Specifically, a CMLM predicts a target sequence T given a source text S and part of the target T masked , where T masked is the target sequence with a random entity being masked. In order to correctly generate the missing part of the sentence, the model needs to condition on both T masked and S. Alternatively, we can calculate the entity's posterior probability using a conditional language model (CLM) instead of a CMLM. In this case, the entity's posterior probability is defined as P CLM (e k | c e k , S) where c e k = g 1 , ..., g i−1 . Note that CLM is only conditioned on the left context.
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
To classify the hallucination and factuality statuses of a given entity, we need to train a discriminator model. We use the K-Nearest Neighbors (KNN) algorithm since it requires no training and makes minimal assumptions about the form of the decision boundary, as a non-parametric method. It also offers adequate interpretability. The KNN classifier is trained using the prior and posterior probabilities as features on our labeled dataset. Since the classifier is used for entity hallucination and factuality assessment, we refer to it as ENTFA. Besides using the prior/posterior probability as features, we also add a binary overlap feature that indicates whether the entity appears in the document. We train two classifiers for hallucination detection and factuality checking tasks respectively.
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
We now propose a factuality-aware training approach for summarization systems that combines our factuality assessment model with the latest offline RL technique.
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
RL for Text Generation Sequence generation of the tokens in the summary text can be viewed as a finite Markov Decision Process (MDP). At each time-step t, the state s t consists of the source text x and the previously generated tokens y <t , s t = (y <t , x). The agent, which is the summarization model, takes an action by generating a new token a t . Depending on the action taken, the agent gets a reward r t = R(s t , a t ) and deterministically transitions to the next state s t+1 = (y <t+1 , x). The probability of each action (i.e., token) is specified by the policy π θ (a t |s t ). The goal of the agent is to maximize the discounted cumulative reward throughout the trajectory: J(θ) = E τ ∼π T t=0 γ t r t . When training the summarization model with human-written reference summaries, we can frame the training process as an off-line RL problem with expert demonstrations (i.e., the reference summaries). In this setting, since we are sampling trajectories from a behavior policy, we need an importance sampling term w t to correct the gradient estimation. Following Pang and He (2021)'s work, we approximate w t with π θ (a t |s t ) and this gives us the following gradient expression of the objective function:
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
whereQ(a t , s t ) = T t ′ =t γ t ′ −t r t ′ is the estimated return from state s t and π b is the behavior policy from which we draw trajectories τ . In our case, π b is the (noisy) summarization dataset.
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
Training with a Factuality-based Reward One problem in the off-line RL setting is that expert demonstrations, which in our case are the reference summaries, are often noisy and contain content that cannot be inferred from the source document. The commonly used teacher forcing training encourages the model to blindly imitate the training data, which leads to model hallucination at inference time (Kang and Hashimoto, 2020).
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
To discourage the model from overfitting to the noise in the training set, we use the predictions from our classifier as factuality reward signals to guide the training of the summarization model. In the off-policy learning stage, we use our factuality classifier to label all the entities in the training set. If an entity is classified by our classifier as "non-factual", we consider it noise and give it a negative reward −r nfe . For factual entities and other tokens, we use the posterior probability from a MLE-trained model as token-level rewards, as in (Pang and He, 2021). Formally, we have:
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
To study entity hallucination and factuality in abstractive summarization, we need annotations of entity-or token-level hallucination. To the best of our knowledge, there is no such dataset available. Therefore, we create a dataset ourselves, which we call the XENT dataset.
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
We 2 annotate 800 summaries generated by BART, which is one of the state-of-the-art abstractive summarization models. The input documents are randomly selected from XSUM test set. We choose XSUM because it is more abstractive than other summarization datasets. We extract 2,838 entities from the 800 generated summaries. We randomly select 30% of the samples as our test set.
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
We manually labeled each entity with one of the following three tags: non-hallucinated, factual hallucination, and non-factual hallucination. First, we extract entities from the given summary using automatic NER tools (Honnibal and Montani, 2017). Then, we check whether each property associated with the identified entity can be directly entailed using the information from the source document. If so, then the property is non-hallucinated. For instance, consider the entity "European Commission President Jean-Claude Juncker" in Table 1. The last name "Juncker" can be directly entailed from Category Source Document Generated Summary
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
Non-hallucinated (...) Tian Tian has had cubs in the past in China, before she came on loan to Edinburgh. (...) The panda enclosure at Edinburgh Zoo is due to close to visitors from Saturday ahead of a possible birth.
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
Edinburgh Zoo's giant panda, Tian Tian, could give birth at the end of the month.
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
The couple, who have been dating since 2011, wed in front of about 10 people in Mazan, Provence -close to where the bride's family has a holiday home. (...) Knightley, 28, announced her engagement to Righton, 29, last year. "Keira was a charming bride, very modest and simple in her attitude, as was James," (...)
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
Oscar-winning actress Keira Knightley and British musician James Righton have married in a small ceremony in France.
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
The city was brought to a standstill on 15 December last year when a gunman held 18 hostages for 17 hours. Family members of victims Tori Johnson and Katrina Dawson were in attendance. (...) Prime Minister Malcolm Turnbull gave an address saying a "whole nation resolved to answer hatred with love". (...) Sydney has marked the first anniversary of the siege at the Waverley cafe in which two women were killed by a gunman in the Australian city.
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
Christopher Huxtable, 34, from Swansea, had been missing since the collapse in February. His body was found on Wednesday and workers who carried out the search formed a guard of honour as it was driven from the site in the early hours of the morning. (...)
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
The body of a man whose body was found at the site of the Swansea Bay Power Station collapse has been removed from the site. Table 2: Examples of four types of hallucinations. In the second example, the nationality of the groom and the country where the wedding took place are not directly stated in the source. According to information online both entities are factual. In the third example, the terrorist attack described in the news took place at a place called "Lindt Cafe" according to Wikipedia. Therefore, "the Waverley cafe" in the generated summary is non-factual. the source document. Therefore, it is not a hallucination. However, the first name "Jean-Claude" and the position information "European Commission President" are not mentioned in the source. In the next step, we need to decide whether these information is factual or not using world knowledge. This often requires external resources such as Wikipedia or Google Search. In this case, "European Commission President" and "Jean-Claude" are both factual. If there is no information found online to prove or disprove the hallucinated entity, it is labeled as non-factual. There is a special case where the entity misrepresents information from the document. For instance, the summary might include a number from the document but that number is actually related to a different event. In this case, the entity is considered as an intrinsic hallucination (Maynez et al., 2020). In this work, we will focus on extrinsic hallucinations, so we discarded all intrinsic hallucinations in our experiments. Table 3 shows the distribution of entities by hallucination and factuality status in our labeled dataset. We show an example for each hallucination type in Table 2.
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
Inter-Annotator Agreement We report Fleiss's Kappa (κ) to access the reliability of agreement between annotators. We compute agreement on a subset of 800 entities and obtain almost perfect agreement (0.80 ≤ κ ≤ 1.00) with κ = 0.809. Following Pagnoni et al. (2021), we also report the percentage µ of annotators that agree with the majority class. We obtain µ = 0.931 of annotators agreeing with the majority class on the fourcategory annotation which shows substantial agreement.
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
Recently, Maynez et al. (2020) released a set of factuality and hallucination annotations for XSUM. For each generated summary, they labeled the hallucinated spans as well as the overall factuality of the summary. Compared with our labeling approach, their annotation has a lower granularity and does not distinguish between factual and non-factual hallucination. Therefore, we have to convert their dataset first before using it for evaluation.
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
To perform entity-level factuality checking on their dataset, we do the following: First, we extract entities from the annotated summaries. For entities that are extracted from factual summaries, we label them as factual entities. For each entity from non-factual summary, if it is inside an extrinsic hallucinated span, then we assume the entity is non-factual. Otherwise the entity is labeled as a factual. This process gives us a new dataset that has the same format as ours for entity-level factuality evaluation. We refer to this new dataset as the MENT dataset.
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
However, it is worth pointing out that the converted dataset is noisy. For instance, in Maynez et al. (2020)'s annotation, the entire generated summary is often labeled as a hallucinated span if it does not capture the meaning of the document well. In this case, the hallucinated span could still contain faithful entities with respect to the source document. This could result in false-positive non-factual entities after the conversion. Therefore, we filter out entities in the extrinsic hallucination span that also appear in the source document.
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
We evaluate our method on entity-level hallucination and factuality classification tasks on XENT and MENT. For each entity in the summary, the model predicts a hallucination label and a factuality label. We will conduct factual and hallucination assessments separately for comparison with the baselines. We compare our method with five baselines models, which are discussed in detail in Section 6.1.
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
In addition to entity-level classification performance, we also evaluate our methods by correlating them against human judgments of factuality. Previous work has collected summary-level judgments of factuality from human annotators, which are then correlated with automatic evaluation measures applied to those summaries. To apply our entity-level method, we use the lowest classifier confidence for the factual class among its entities as the factuality score for the entire summary. We evaluate correlation on two datasets by Pagnoni et al. (2021) and .
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
To evaluate our factuality-aware training approach proposed in Section 3.3, we train a summarization model with factuality rewards and evaluate model's predictions on XSUM test set. To evaluate the faithfulness of generated summaries, we use automatic faithfulness evaluation tools FEQA (Durmus et al., 2020) and DAE (Goyal and Durrett, 2020) 3 . We also calculate ROUGE scores, and the percentage of n-grams and percentage of entities in the generated summaries that are not found in the source document (ENFS). The percentage of novel n-grams   
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
Baselines We compare with five baseline methods:
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
(1) The overlap-based method checks the word overlap between the summary and the source document. In our case, we check whether a given entity in the generated summary also exist in the source document. If it does not, the entity is clas-sified as both hallucinated and non-factual.
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
(2)
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
The synonym-based baseline extends the overlapbased baseline by checking the overlap of summary synonyms and source synonyms. See Zhou et al. (2020) for more details.
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
(3) The alignmentbased baseline is based on the unsupervised word alignment method SimAlign by Jalili Sabet et al.
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
(2020). SimAlign extracts word alignments from similarity matrices induced from pretrained embeddings. In our task, we treat all unaligned entities in summaries as hallucinated and non-factual. (4) The LM-based method is proposed by Filippova (2020). The LM-based method uses LM and CLM to compute the token's prior and posterior probability. In Filippova (2020)'s work, they compare the value of p prior and p pos . If the generated token does not match the reference and p prior is greater than p pos , the token is classified as hallucinated.
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
Since we are evaluating the generated summary but not the reference, we modify their method to the following: if the entity is not found in the source and p prior > p pos , then the entity is classified as non-factual and hallucinated. (5) Zhou et al. (2020) frame the hallucination detection task as a sequence labeling task. They train a hallucination labeling model on synthetic data. We adapt their model to our task by finetuning their model on XENT.
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
Evaluation Results on XENT Table 4 shows the evaluation results of our classifiers and baselines in terms of both entity factuality and hallucination status classification. The results show that our approach outperforms five baselines on the factuality classification task. To show that our model is statistically better than the baselines, we run a 10-fold cross-validated paired t-test comparing our model with five baselines. The results show that our model is better than the baseline models with p-value less than 3.27e − 5. On the hallucination detection task, the overlap-based and synonym-based baselines achieve relatively high accuracy. However, these methods cannot distinguish between factual and non-factual hallucinations. This is the reason for their performance degradation on factuality classification task. For hallucination classification, the reason computing word overlap with the source does not completely solve the hallucination detection problem is that hallucination is defined based on the semantic relationship between the source and the summary. There can exist words that are not in the source document but which can nevertheless be inferred from it.
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
Overlap    Table 5 shows the evaluation results on MENT. ENTFA are learned on our annotated training set with k set to 20. The performance of all models is lower on this dataset. This may be due to fact that the converted dataset is noisier than the XENT dataset (see Section 4.2). For the factuality classification task, our model outperforms five baseline models. This demonstrates the generalizability of our approach.  
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
Baselines We compare our approach with four baselines: a teacher forcing trained summarizer (MLE), a RL-based summarizer (RL) (Pang and He, 2021) and a summarizer trained with the loss truncation technique from Kang and Hashimoto (2020). We also replace our factuality assessment model ENTFA with Filippova (2020)'s approach (LM-based) for entity factuality labeling as another baseline model (see Section 3.3). Table 7 shows the evaluation results on XSUM. The results show that our approach outperforms all baselines with fewer non-factual entities and higher faithfulness scores. Note that our approach has the lowest ENFS rate while having the highest percentage of factual hallucinations. Compared with the loss truncation baseline, our method also produces more novel n-grams. These show that our method does not improve the factuality of the model by simply making the model more extractive. Figure 1 shows the factuality and abstractiveness trade-off curves of our model compared to the loss truncation baseline. At the same level of ROUGE performance, our method can obtain a higher factuality score. This further proves that our model can generate both factual and high-quality summaries compared with the loss truncation baseline.
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
To explore the effect of each feature, we conduct an ablation study by training the KNN classifier with fewer features. The results are illustrated in Table 8 and show that all the proposed features are useful. For factuality classification, The performance w/o posterior drops significantly from 81.82 to 70.30. This result suggests that the posterior probability is crucial for factuality classification. For hallucination classification, the overlap-based feature has the most significant impact on model performance. In Figure 2b, the KNN classifier separates the factual and non-factual entities with clear boundaries. A large part of the factual hallucinated entities are correctly identified by CMLM XSUM with relatively high posterior probabilities. This explains our model's superior performance on factuality checking. The top and right histograms in Figure 2b show the entity distribution over prior and posterior probability value respectively. As shown in 2b's histogram, factual entities have significantly higher posterior probability than that of non-factual entities on average. Figure 3 shows histograms of the prior and posterior probabilities of entities from MLM and CMLM XSUM , separated by their class (i.e., whether they are hallucinated and/or factual). Nonhallucinated entities have higher posterior probability than factual and non-factual hallucinations on average.
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
In this paper, we investigate the hallucination and factuality problems in abstractive summarization. We show that about 30% of entities generated by state-of-the-art summarization model are hallucinated. More interestingly, more than half of the hallucinated entities are factual with respect to the source document and world knowledge. We propose a novel method based on the entity's prior and posterior probabilities according to masked language models. Our approach outperforms five base-  Table 7: Comparison of different summarization models. Results are evaluated on XSUM's official test set. "% Factual Ent" and "% Factual Hal" are the percentage of factual entities and factual hallucinations classified by ENTFA model respectively. "% ENFS" is the percentage of entities in generated summary that not found in source document. For the loss truncation baseline, c is the percentage of data to be dropped.    line models on both factuality classification and hallucination detection tasks on human-annotated datasets. In addition, using our classifier as a reward signal vastly improves the factuality of summarization systems. Our approach is limited to entity-level hallucination and factuality classification. In the future, we are interested in extending our work to arbitrary text spans. 
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
Before annotating the dataset at full-scale, we conducted a pilot study with the annotators on a small evaluation set that contains 10 document and summary pairs. We then discussed with the annotators and had them explain the labels they were given to ensure they fully understood the task and followed the guidelines. The guidelines can be summarized as follows:
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
(1) Read the source documentation and generated abstract. If the article is incomprehensible (e.g. too short or in a language other than English), mark it as corrupted.
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
(2) For each entity in the summary (identified by NER tool), check whether the entity can be directly entailed in the summary context using only the information within the source document. If the answer if yes, label the entity as non-hallucinated. If the entity has multiple properties, annotate each property separately.
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
(3) If the source does not contain sufficient information to entail the entity, use Wikipedia or Google Search to determine the factuality of the entity. If no information can be found to prove or disprove the factuality of the entity. Label it as non-factual hallucination.
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
(4) If the entity is mentioned in the source document, but it is used in the wrong context and misrepresents information from the document. Label the entity as intrinsic hallucination.
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
We also ask the annotators to mark and annotate entities missed by automatic NER tools. We will then update the identified entities to ensure that the samples are consistent for all annotators. Annotators are paid 20$ an hour for their work, which is above the minimum wage in their country of residence. Table 9 shows the patterns of hallucinated entities. For factual hallucinations, Person, GPE, and ORG are the three most common types. Among nonfactual hallucinations, Date is the most common type (31.65%). Cardinal numbers are also easily hallucinated by summarization model. Note that the proportion of Date and GPE type of entities in non-factual hallucinations is much higher than their proportion in all entities.   (Paszke et al., 2017). For the CMLM training, we initialize the model with the checkpoint of the large BART model. The max sequence length is set to 1024 for both the encoder and decoder modules. We fine-tuned the model for 15,000 steps with the warm-up steps set to 500. We use the standard cross-entropy loss as our objective function with 0.1 label-smoothing (Szegedy et al., 2016). The Adam optimizer (Kingma and Ba, 2015) with ϵ = 1e-8 and an initial learning rate 3e-5 are used for training. The dropout rate in each layer is set to 0.1. These hyperparameter values are based on the recommended values from the fairseq (Ott et al., 2019) library All experiments are conducted on 4 Tesla V100 GPUs with 32GB of memory.
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
In the off-line RL experiment, we initialize the model using the BART large model finetuned on XSUM dataset 4 . The discount factor γ is set to 1 and the learning rate r is set to 1e − 5.
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
We update the model for 30,000 steps in total with 1000 warm-up steps. We use polynomial decay to update the learning rate after each training step. No reward-shaping is used.
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
To make the training more stable, we use another policy networkπ θ to compute the importance weight w.π θ is kept as a slow copy of π θ with the same model architecture. We use Polyak updates to slowly update the weight ofπ θ in the direction to match π θ every step. The update rate ofπ θ is set to 0.01.   Table 10 shows the three-class classification results of our model on XENT dataset. Since we are the first work (to the best of our knowledge) that distinguishes between factual and non-factual hallucinations, we did not have a baseline model to compare with right now. We compare with other models separately in terms of factuality and hallucination classification in Section 6.1.
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
Training Data  Figure 4 shows the evaluation results of summarization models trained on these datasets. We can see that the model generates fewer factual entities as the training set gets noisier. Also, it shows that ROUGE score is not a favorable metric in terms The y-axis on the right indicates ROUGE-1 scores. X-axis = 0 and x-axis = 1.0 means that the model is trained on 50k clean samples and 50k noisy samples respectively; x-axis = 0.5 represents the model trained on a mix of 25k clean samples and 25k noisy samples. X-axis = 2.0 represents a model that is trained on 100k noisy samples. All models are tested on XSUM's official test set. We observe a similar trend with the PEGASUS model ( Figure 5).
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
of factuality evaluation. Since with the training set size fixed, the model seems to achieve higher ROUGE score at the expense of entity factuality. In addition, this indicates that if the system is optimized only for ROUGE, they may inadvertently harm factual consistency. We also observe that the word overlap method predicts much lower entity factuality rate than ENTFA. This is due to the fact that the word overlap method cannot identify factual hallucinations and introduce many false-negative samples. To verify this, we extracted all entities from summaries generated by the model trained on 50k noisy samples (x-axis = 1.0). Among these entities, there are 7,358 entities that do not appear in the source but are predicted as factual by our model. We find that 50.5% of these entities can be found in the reference summary. As a contrast, only 12.7% entities predicted as non-factual by our model can be found in the reference. Figure 5 shows the evaluation result of PEGA-SUS model (Zhang et al., 2020) follows the evaluation set up in Section A.5. Both figures show a similar trend that the models get higher ROUGE score when trained on noisier dataset with the cost of generating more non-factual entities. Compared with BART model, PEGASUS gen- erates more hallucinated entities and has higher ROUGE score overall. For instance, when both trained on 50k clean data, PEGASUS has ROUGE-1 score 0.450 compared with BART's 0.406. The predicted factual entity rate for PEGASUS and BART is 84.79% and 91.81% respectively. This may be due to the fact that PEGASUS is pretrained on a much larger corpus than BART. We leave the study of this phenomenon to future work.
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
A.6 Where Does the Model Learn to Hallucinate? Table 3 shows that 30% of the entities in the summaries generated by BART are hallucinated, including 15% factual hallucinated entities. To generate factual hallucinated entities, the summarization model needs to integrate background knowledge into the summary. One interesting problem is investigate where the model learns that knowledge. Since the BART is pre-trained on a large text corpus and fine-tuned on XSUM, the knowledge of hallucinated entities could come from either the pre-training corpus or the XSUM training set. To investigate this, we trained a separate CMLM on the CNN/DM dataset. Figure 6 shows the entity distribution from the two CMLM models. For non-hallucinated entities, the distributions are similar; for factual hallucinations, we can find that a large portion of them has very low posterior probabilities under CMLM CNN/DM , but high posterior under CMLM XSUM . This pattern suggests that the knowledge of many factual hallucinations comes from the XSUM training set.
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
We define σ(e k ) = log P CMLM XSUM (e k ) P CMLM CNN/DM (e k ) . If σ(e k ) ≥ 0, it suggests that CMLM XSUM is more confident that e k is factual than CMLM CNN/DM . For a factual hallucination e k , we can infer that the knowledge of e k is in XSUM if σ(e k ) is large. To further verify this, we retrieve the 10 most similar documents from XSUM and CNN/DM for each factual hallucinated entity using TF-IDF. Then, we count the number of times each entity appears in those similar training samples. For entities with σ(e k ) ≥ 5, the average number of appearances is 2.19 on XSUM and 0.77 on CNN/DM. For entities with σ(e k ) ≤ 0, the average number of appearances becomes 2.85 and 2.46 on XSUM and CNN/DM respectively. This further confirms that the knowledge of factual hallucinations with large σ(e k ) comes from XSUM.
Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization
A.7 Compare with Filippova (2020)'s Work Filippova (2020)'s work on data-to-text generation shows that low posterior probability from a CLM during decoding indicates hallucination. Take the summarization model as an example, if an entity is generated with very low posterior probability, it is likely that the generated entity is hallucinated and non-factual. However, compared with CMLM, CLM has more uncertainty during decoding since the right context of the entity is not determined. The uncertainty of the CLM comes from both content selection (text content and structure) and lexical choice (Xu et al., 2020). For CMLM though, the uncertainty is mostly reduced to the latter. Figure 7 show the entity posterior probabilities from CLM and CMLM model. As shown in the figure, we can find that most factual entities (blue points) are above the x = y line. This means CMLM gives more certainty to the same factual entity than CLM. The ROC curve in Figure 8 further shows this. As the lines get closer to the origin, the threshold becomes larger, and CMLM has a higher TPR than CLM. This means CMLM will classify more entities as factual. The higher AUC value of CMLM further demonstrates that CMLM is a better choice for factuality checking than CLM. 
Improving Faithfulness in Abstractive Summarization with Contrast Candidate Generation and Selection
Abstractive Summarization is the task of producing a concise and fluent summary that is salient and faithful to the source document(s). Datadriven, neural methods (Rush et al., 2015;Nallapati et al., 2016;See et al., 2017), and the more recent, pretrained transformer language models (Vaswani et al., 2017;Devlin et al., 2019;Liu and Lapata, 2019), have shown improvements in the fluency and salience of generated summaries.
Improving Faithfulness in Abstractive Summarization with Contrast Candidate Generation and Selection
However, less progress has been made on improving the faithfulness of the generated summaries, that is, producing a summary that is entailed by the information presented in the source document. Despite the increased level of performance under automatic metrics such as ROUGE * Most of the work done while the authors were at Google.
Improving Faithfulness in Abstractive Summarization with Contrast Candidate Generation and Selection
Source: He was re-elected for a second term by the UN General Assembly, unopposed and unanimously, on 21 June 2011, with effect from 1 January 2012. Mr. Ban describes his priorities as mobilising world leaders to deal with climate change, economic upheaval, pandemics and increasing pressures involving food, energy and water... Unfaithful Summary: The United Nations Secretary-General Ban Ki-moon was elected for a second term in 2007. Our Summary: The United Nations Secretary-General Ban Ki-moon was elected for a second term in 21 June 2011. Table 1: An example unfaithful summary. It suffers from extrinsic hallucination, where information not present in the source document was generated. Our method attempts to correct the unfaithful summary by replacing "2007" with an entity from the source with compatible semantic type (i.e. DATE). (Lin, 2004) or BERTSCORE , current state of the art models (Liu and Lapata, 2019;Lewis et al., 2020) produce summaries that suffer from intrinsic and extrinsic hallucinations -the fabrication of untruthful text spans containing information either present or absent from the source (Maynez et al., 2020). Table 1 shows an example of such summary, generated by BART (Lewis et al., 2020), an auto-regressive, transformer-based sequence-tosequence model. The article describes an event where the former UN-Secretary-General Ban Ki-Moon was re-elected for a second term. The model hallucinates "2007", which never appears in the source document, leading to inconsistency with the correct date of the event presented.
Improving Faithfulness in Abstractive Summarization with Contrast Candidate Generation and Selection
In this work, we focus on the problem of correcting such hallucinations as a post processing step 1 . A post processing correction step allows us to rely on the fluency of the text generated by SOTA systems, that gain from huge pretrained models and large fine-tuning datasets, and correct it using small 5936 amounts of automatically generated training data.
Improving Faithfulness in Abstractive Summarization with Contrast Candidate Generation and Selection
Under the setting where a large fraction of ground truth summarization data is hallucinated, as we show in Table 2, we study the method of contrast candidate generation and selection. In the generation step, we replace named entities in a potentially hallucinated summary with ones with compatible semantic types that are present in the source, and create variants of candidate summaries. In the selection step, we rank the generated candidates with a discriminative model trained to distinguish between faithful summaries and synthetic negative candidates generated given the source. We experiment on a range of RNN-and transformer-based abstractive summarization models. Our preliminary results on the XSum corpus (Narayan et al., 2018a), which contains substantial presence of hallucinated ground truth examples, show the effectiveness of our method in correcting unfaithful summaries with extrinsic hallucinations.
Improving Faithfulness in Abstractive Summarization with Contrast Candidate Generation and Selection
Our main contributions are as follows. First, our work is the first to study the effectiveness of contrast candidate generation and selection as a model-agnostic method for correcting hallucinations, under the setting where a large fraction of ground truth summarization data suffers from hallucinations. Second, we validate our method on various neural summarization systems trained on XSum, and provide detailed analysis on the typical types of hallucinations from each system.
Improving Faithfulness in Abstractive Summarization with Contrast Candidate Generation and Selection
Our proposed method is built on the observation that a large fraction of extrinsic hallucinations happen on named entities and quantities. Table 2 shows the human analysis by Maynez et al. (2020) on the hallucinations of 500 randomly sampled gold summaries from the XSum corpus . We break down each category and annotate the proportion of hallucinations that happen on entity and number/quantity spans. As Maynez et al. (2020) further show that the hallucinations in training data translate to similar issues for the generated outputs across different summarization models, we want to study a modelagnostic, post-processing method that can correct such entity and quantity hallucinations. We frame the problem as a correction task and make it conceptually a less complex problem than summarization. Modeling correction as a standalone task would  The "%" column shows the % of intrinsic and extrinsic hallucinations annotated by Maynez et al. (2020). We analyzed the % of hallucinations on entities and numbers/quantities, and show the % out of all 500 summaries in the right two columns.
Improving Faithfulness in Abstractive Summarization with Contrast Candidate Generation and Selection
require less training data, which becomes crucial when a large proportion of ground truth summarization data suffer from hallucinations, and inherit the fluency of data intensive SOTA models.
Improving Faithfulness in Abstractive Summarization with Contrast Candidate Generation and Selection
From a model-generated summary, we first identify any potentially hallucinated entities or quantities by checking whether entities with similar surface forms have appeared in the source document. We use a neural Named Entity Recognition (NER) system from the Stanza NLP toolkit (Qi et al., 2020) trained on the OntoNotes corpus (Weischedel et al., 2013) to extract named entities of different semantic types from the source document and summary. Each named entity present in the summary is replaced with a different entity present in the document with the same NER label. This gives us different variants of the original summary with the same level of fluency , but not necessarily faithful.
Improving Faithfulness in Abstractive Summarization with Contrast Candidate Generation and Selection
For the candidate selection step, we want to identify the best candidate among the variants generated in the previous step as the final output summary.
Improving Faithfulness in Abstractive Summarization with Contrast Candidate Generation and Selection
As the contrast candidates vary in no more than a few tokens from the original summary, it requires a model with more delicate local decision boundaries (Gardner et al., 2020) to select the correct candidate. For example, we observe that MNLI models (Williams et al., 2018) fail to produce satisfactory results.
Improving Faithfulness in Abstractive Summarization with Contrast Candidate Generation and Selection
To create training data for that purpose, we sample examples from the XSum training set where all entities in the ground truth summary appear in the source document. We then follow the same procedure in the generation step, and produce unfaithful variants from the ground truth summary by replacing entities with others that have the same se-mantic type but different surface form in the source text. With the ground truth and synthetic negative summaries, we train a text classifier with a discriminative objective to score and rank the variants of the summaries.
Improving Faithfulness in Abstractive Summarization with Contrast Candidate Generation and Selection
We use BART (Lewis et al., 2020) plus a linear layer as our classification model. We adopt a similar learning objective to contrastive learning (Khosla et al., 2020). For each pair of positive and negative summary candidate, we use cross entropy loss L XE to handle the correctness of the label predictions. We add a margin ranking loss term L RANK to encourage the model to assign higher probability to the positive than the negative candidate. The margin γ is a tunable hyperparameter in training.
Improving Faithfulness in Abstractive Summarization with Contrast Candidate Generation and Selection
During test time, we use the trained model to score the generated contrast candidate summaries, as well as the original version generated by the summarization model. We take the candidate with the highest score as the final summary.   in the source text. Our model decides to keep the original summary in the rest 48.3%.
Improving Faithfulness in Abstractive Summarization with Contrast Candidate Generation and Selection
We first verify that our method does not hurt the fluency and salience of the generated summaries, for which we assume ROUGE (Lin, 2004) and BERTSCORE  are suitable metrics. We report the results in Table 3. We observe though both the baseline and our method do well in both ROUGE and BERTSCORE, our method trails behind in both metrics slightly. This is due to the existence of extrinsic hallucinations in the ground truth summary, and the model manages to generate a part of the hallucinations, and gets incorrectly rewarded by such.
Improving Faithfulness in Abstractive Summarization with Contrast Candidate Generation and Selection
To test whether our correction method can improves the faithfulness of the summaries, we evaluate the summaries with FEQA (Durmus et al., 2020), a QA-based metric for summary faithfulness. Given a summary, FEQA automatically generates questions on noun phrase and named entity spans in the summary, and uses a pretrained QA model to verify if the answer derived from the source document exact-matches the span in the summary.
Improving Faithfulness in Abstractive Summarization with Contrast Candidate Generation and Selection
We run FEQA and compute the macro-averaged percentage of questions answered correctly for each of the 1510 summaries that our system made corrections to, and report the results in Table 3. The results suggest that the corrected summaries present statistically significant improvements over the original ones (p < 0.001, with a two-tailed, paired t-test). Table 4 shows the human evaluation results on the 95 randomly sampled subset of changed summaries. Two expert annotators assign each summary into three faithfulness categories and adjudicate the decisions. Additional annotations from   a third expert is then used to calculate the interannotator agreement. As the results show, our model is able to improve the faithfulness of the summaries, but at the cost of incurring intrinsic hallucinations on mistakes, which we will discuss more in detail in section 4.2.  (Rothe et al., 2020). Our system achieves consistently high level of precision across models. The system achieves high relative recall with respect to the % of entity and quantity hallucinations among all hallucinations. As our method only targets entities and quantities, the overall recall varies by the typical type of hallucinations each summarization system makes. We also observe while our method achieves high recall on models with lower ROUGE and BERTSCORE, the recall drops on pretrained models such as BERTS2S. This is potentially due to the decreased percentage of entity/quantity hallucinations exist in generated summaries from the models with pretraining.
Improving Faithfulness in Abstractive Summarization with Contrast Candidate Generation and Selection
As our method detects and corrects extrinsichallucinated entities, naturally any entities replaced wrong would introduce intrinsic hallucinations in the changed summary, as indicated by the results in Table 4. To speculate why the mistakes happen, we analyzed the typical mistakes by the model, and listed a few representative examples in Table 5. For example, our method could not find the correct replacement for a hallucinated entity when no such one exists in the source text. We observe that the models with pretraining, such as BERTS2S, (Rothe et al., 2020) and BART, suffer from the issue by most, as they tend to be affected by artifacts/priors from the pretraining process.
Improving Faithfulness in Abstractive Summarization with Contrast Candidate Generation and Selection
From the observation that models often hallucinate entities with no correct replacement in the source, we suspect that solving entity faithfulness alone does not guarantee the faithfulness of the summary. In the last example from Table 5, the BERTS2S system correctly identifies that three fugitives are involved in the event described by the source text, even though the number "three" has never been explicitly mentioned in the source context in any surface forms. Furthermore, statistics provided by Maynez et al. (2020) show that abstractive summarization models often produces factual statements,
Improving Faithfulness in Abstractive Summarization with Contrast Candidate Generation and Selection
i.e. verifiable in the real world independent of the source text. Such findings imply that identifying hallucinations often requires more complex objectives such as commonsense reasoning and knowledge retrieval. The solution we propose here that focuses only on entites and quantities would likely be insufficient to solve the entire problem.
Improving Faithfulness in Abstractive Summarization with Contrast Candidate Generation and Selection
We study contrast candidate generation and selection as a method to apply post-hoc fixes to extrinsically hallucinated summary on entities and quantities, under the setting where the summarization dataset suffers from intrinsic and extrinsic halluci-nations. We conduct our experiments on the XSum dataset, and show that our method is able to correct extrinsic hullucinations, but incurs a small fraction of intrinsic hallucinations on mistakes. We also provide detailed analysis and discussions on the capabilities and limitations of our method. We hope our findings in the paper will provide insights to future work in this direction.
Improving Faithfulness in Abstractive Summarization with Contrast Candidate Generation and Selection
Ranking generated summaries by correctness: An interesting but challenging application for natural language inference. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2214Linguistics, pages -2220 
Improving Faithfulness in Abstractive Summarization with Contrast Candidate Generation and Selection
For our contrast candidate selection model, we use a pretrained BART base model. We add a linear layer over the max pooled embedding, and the classification model is expected to output a label between ["FAITHFUL", "HALLUCINATED"]. For all our experiments, we use the following set of hyper-parameters: r = 1e − 5, margin γ = 0, number of training epoch= 3.  
Improving Faithfulness in Abstractive Summarization with Contrast Candidate Generation and Selection
We use bootstrapping to estimate the confidence interval for the expert annotation presented in Table 4. For each faithfulness category on the two systems, we regard the adjudicated annotation as ground truth, and label the individual instance as the true positive (TP), false negative (FN), true negative (TN) and false positive (FP) respectively according the annotations from the third expert. We re-sample the 95 instances with replacement for 1,000 times. We estimate the adjusted mean and 95% confidence interval from the mean and standard deviation of the sampled distribution of (TP + FN).
Improving Factual Consistency in Summarization with Compression-Based Post-Editing
Text summarization aims to compress a long document(s) into a short and fluent form that preserves salient information. State-of-the-art models, however, are often not factually consistent with the input they are conditioned on (Maynez et al., 2020;Fabbri et al., 2022). While recent modeling techniques have been proposed to improve factual consistency (Nan et al., 2021;Kang and Hashimoto, 2020;Aralikatte et al., 2021), a model-agnostic approach is to post-edit the summaries.
Improving Factual Consistency in Summarization with Compression-Based Post-Editing
Prior work in post-editing for factual consistency has focused on swapping inconsistent entities with those in the input Lee et al., 2022), including by reranking the entity-replaced   (Chen et al., 2021) does not modify the errors (red) if a suitable entity replacement is not found, revisionbased editing (Adams et al., 2022) performs more extensive changes, deletion (Wan and Bansal, 2022) removes errors but affects grammaticality (orange), while ours returns a well-formed summary through compression.
Improving Factual Consistency in Summarization with Compression-Based Post-Editing
summaries (Chen et al., 2021), autoregressive approaches that learn to rewrite and remove perturbations from the input (Cao et al., 2020;Zhu et al., 2021;Adams et al., 2022), or deletion-based editing of references (Wan and Bansal, 2022). We provide example outputs of these approaches that demonstrate potential downsides in Table 1. For summaries in datasets such as XSum (Narayan et al., 2018) that contain extrinsic entity errors in up to 70% of reference and generated summaries (Maynez et al., 2020), a suitable entity replacement is often not available from the source.
Improving Factual Consistency in Summarization with Compression-Based Post-Editing
To address such errors, we propose a post-editing model that performs controlled compression: given input text and a set of (factually incorrect) entities, it generates a compressed output with the specified entities removed, as demonstrated at the bottom of Table 1. To synthesize a training set for this model, we use existing sentence compression data to first train a perturber model that maps a compressed sentence and a list of entities to an uncompressed sentence containing those entities. We then apply Figure 1: Overview of our compression-based approach. A perturber model is trained to produce an uncompressed sentence with inserted entities, conditioned upon the compressed sentence and an appended list of entities. The perturber is applied to a dataset's reference summaries to create data to train a post-editor that takes the document (SRC) and a summary with entities to remove surrounded by special tokens, and removes the erroneous words.
Improving Factual Consistency in Summarization with Compression-Based Post-Editing
the perturber model to insert entities on a subset of the target dataset whose summaries contain only named entities also found in the input. Our posteditor is then trained in the reverse direction, conditioning upon the input article and the perturbed reference summary, with entities to remove marked with special tokens, to produce the compressed summary, as illustrated in Figure 1.
Improving Factual Consistency in Summarization with Compression-Based Post-Editing
Our contributions are the following: 1) We propose a compression-based method for summary post-editing that removes extrinsic entity errors, improving entity precision with respect to the input by up to 25% along with improvements in other factual consistency metrics while retaining informativeness according to automatic analysis. 2) We show that this method can be applied on top of a rewriting-based post-editor, improving entity precision by up to 38% overall with a small decrease in ROUGE score. 3) We perform an extensive comparison of prior post-editing methods across two datasets and six summarization models to better understand the trade-offs between factual consistency, informativeness, and grammaticality. Models are made publicly available: https://github.com/salesforce/CompEdit.
Improving Factual Consistency in Summarization with Compression-Based Post-Editing
SpanFact We implement a variation of the autoregressive model from . We train a BART-large (Lewis et al., 2020) model to take the source document and the summary with entity slots masked and fill in the masked slots. We also train the model on the data subset whose summaries contain only named entities found in the input, which we call SpanFact-c. CCGS We apply the model from Chen et al. (2021) that generates candidate summaries by enumerating all ways to replace summary entities with similartyped input entities and training BART with a clas-sification layer to re-rank these summaries. ReDRESS We apply the approach from Adams et al. (2022) for revising clinical reference summaries to news summarization. The approach consists of two stages: 1) a perturber learns to corrupt a summary by using entity swaps between the input and a retrieved set of entities, span deletion, and shuffling as training data. 2) The perturber is then applied to the reference summaries to create training samples for a reviser that learns to remove errors through contrastive learning. In contrast, our approach focuses on more controlled perturbations and revisions to remove and compress rather than rewrite, which may insert errors. FactPegasus We apply the deletion-based corrector component of Wan and Bansal (2022). This method removes extrinsic entity error tokens and surrounding words based on manually-defined rules over the dependency parse of the summary, which may introduce grammatical errors.
Improving Factual Consistency in Summarization with Compression-Based Post-Editing
We train a post-editor in two steps on sentence compression data from Filippova and Altun (2013), where the uncompressed sentence can be viewed as the summary containing information not present in the compressed version. Example input and output for these two steps are shown in Figure 1. We first train a BART-large perturber model conditioned upon a compressed sentence and entities present in the longer sentence but not present in the compressed one to produce a longer sentence. In other words, we maximize the following probability: P (Uncompressed | Compressed, Ents). Then, for a given data point in the summarization dataset, we select each of one, two, and three entities from the source not present in the reference and condition upon those entities and the reference summary to produce a longer, perturbed version containing those entities. Varying the number of en-  (Chen et al., 2021) 64  tities inserted mirrors differing levels of editing required. Note that while the entities inserted are not extrinsic errors since they appear in the source, they are inserted out-of-context since the perturber does not condition upon the source document. We then train our BART-large model post-editor model, which we call CompEdit, to produce the original reference summaries conditioned upon the source document and these perturbed summaries, with special tokens surrounding the entities that should be removed. In this setup, the post-editor learns to remove related, out-of-context entities during training. We only train on references for which all named entities were found in the source and thus maximize the following: P(Gold summary | source, special-token perturbed summary). While a post-editor could be trained directly from sentence compression data or on summarization data without access to the source, such models resulted in a degradation of summary salience. During inference, we surround extrinsic entity errors, as determined by named entity overlap with the input, with special tokens to signal the model to remove them.
Improving Factual Consistency in Summarization with Compression-Based Post-Editing
We evaluate the above approaches on the XSUM (Narayan et al., 2018) and CNN/DM (Hermann et al., 2015) datasets. We apply BART-large as the base summarization model, and post-editing models are applied to this base model's output. We also report results for BART-large trained on a data subset in which all summary named entities are found in the source, which we call BART-c.  CNN/DM results, are found in the Appendix.
Improving Factual Consistency in Summarization with Compression-Based Post-Editing
To show the generalizability of our results to post-editing models other than BART, we test the best post-editors on UniLM (Dong et al., 2019) and BottomUp (Gehrmann et al., 2018) outputs (additional models in the Appendix). Zhu et al. (2021) released the output of their post-editor FASumFC on these outputs but not their trained model. This post-editor is a seq2seq denoising model trained with entity swaps for inserting inconsistencies and backtranslation to create paraphrases.
Improving Factual Consistency in Summarization with Compression-Based Post-Editing
We evaluate using standard ROUGE-1/2/L (R-1/2/L) and include a variation called R1-c that evaluates R1 on the reference summaries with entities not found in the input removed from the summary. We include the percentage of the base model summaries that are edited by the post-editor (Edit%) and the following metrics: E-P src (E-R ref ) measures the percentage of entities in the generated summary (reference) present in the input (generated summary). E-P src as a metric performs on par with model-based, token-level metrics (Zhou et al., 2021;Cao et al., 2022). The subset of data without named entity errors has an E-P src of 100.  
Improving Factual Consistency in Summarization with Compression-Based Post-Editing
Results of applying post-editing models to BART on XSum are shown in Table 2, and examples, and results on CNN/DM, are in the Appendix. We note that BART-c improves across all factual consistency metrics compared to BART at the cost of ROUGE and other informativeness metrics. We find that SpanFact does not improve factual consistency, as this model is trained to fill in entity masks on the original, noisy dataset, but SpanFact-c improves factual consistency. While CCGS does improve factual consistency slightly, the model only edits a small percentage of the summaries (Edit% of about 15). Other post-editors make a more realistic number of edits, considering that over 70% of XSum reference summaries may contain factual inconsistencies (Maynez et al., 2020).
Improving Factual Consistency in Summarization with Compression-Based Post-Editing
ReDRESS performs better than SpanFact-c on factual consistency and also E-R ref , which aligns with its rewriting objective that can insert semantically relevant entities. FactPegasus improves entity precision by definition (it is not 100 due to differences in entity processing), but it is the only post-editor that decreases grammaticality by a noticeable margin. Our CompEdit model improves factual consistency, and we see further improvements when applying our model on top of the Re-DRESS corrector; ReDRESS as a first-stage corrector can insert suitable replacement entities while the second-stage compressor may remove any remaining entity errors. CompEdit does not remove all entity errors in the summaries, and a qualitative inspection revealed that common world knowledge tokens, such as names of world leaders, were often left unchanged. Recent work has also noted the presence of extrinsic world knowledge errors (Cao et al., 2022), and we leave a larger study of such artifacts to future work. We note that vanilla ROUGE scores do decrease along with improvements in factual consistency. However, R1-c actually shows improvements when applying FactPegasus or Com-pEdit, showing that much of the loss in vanilla R1 is due to factual inconsistencies in the references.
Improving Factual Consistency in Summarization with Compression-Based Post-Editing
We show the results of applying the above posteditors on non-BART models in Table 3. We find large improvements over Zhu et al. (2021). We see a gain in R1-c and entity precision when applying CompEdit to the pretrained UniLM, showing the benefits of our compression-based approach when the underlying model contains relatively highquality summaries. ReDRESS shows a large performance increase on BottomUp; its ability to completely rewrite the summary allows it to improve the non-pretrained, lower-quality summaries that require editing beyond just compression. There is also a benefit from the combination of ReDRESS and CompEdit in terms of entity precision for the BottomUp model, which has been shown to contain a high proportion of entity errors and inconsistencies (Huang et al., 2020;Pagnoni et al., 2021).
Improving Factual Consistency in Summarization with Compression-Based Post-Editing
Finally, we show the manually-annotated results of post-editing a non-pretrained model from the CNN/DM-based FactCC (Kryscinski et al., 2020) test dataset in Table 4. We compare with previously-reported results: an autoregressive posteditor trained on entity swaps and backtranslated paraphrases (Cao et al., 2020), and a method that substitutes entities from retrieved source sentences (Lee et al., 2022). As extrinsic entity errors, for which CompEdit and FactPegasus are designed, constitute only a small proportion of errors in this dataset, we only include ReDRESS + CompEdit. CompEdit was applied to the ReDRESS outputs containing extrinsic named entity errors, 2.5% of edited summaries, all of which were then labeled factually consistent. We see a large improvement in consistent summaries and similar trends in the benefit of ReDRESS and adding CompEdit as the above results on the non-pretrained BottomUp model.
Improving Factual Consistency in Summarization with Compression-Based Post-Editing
In this work, we propose a sentence-compressionbased post-editing model to improve factual consistency in summarization while maintaining the informativeness and grammaticality of the resulting summaries. We show that this model can be used in tandem with a post-editor that performs extensive rewriting for further improvement, especially in pretrained models and datasets with a high proportion of entity errors. In future work, we plan to build on these models by studying the role of dataset artifacts in error correction and addressing unfixable summaries.
Improving Factual Consistency in Summarization with Compression-Based Post-Editing
We train and test our models on publicly available news summarization datasets. Political and gender biases may exist in these datasets, and thus models trained on these datasets may propagate these biases. Furthermore, while our models are not language-specific, we only study English summarization.
Improving Factual Consistency in Summarization with Compression-Based Post-Editing
When used as intended, these post-editing models can help eliminate some factual inconsistencies in model summaries. However, these models do not remove all factual inconsistencies, and thus much care should be taken if one wants to use such models in a user-facing setting.
Improving Factual Consistency in Summarization with Compression-Based Post-Editing
The experiments in this paper make use of A100 GPUs. We used up to 8 GPUs per experiment, and the experiments may take up to a day to run. Multiple experiments were run for each model, and future work should experiment with distilled models for more lightweight training. 
Improving Factual Consistency in Summarization with Compression-Based Post-Editing
Results of applying post-editing models to BART on CNN/DM are shown in Table 5. BART-c decreases performance in factual consistency on CNN/DM, perhaps due to the reduction in training data points; CNN/DM is largely extractive and encourages factually consistent as-is. This is reaffirmed by the small gap between SpanFact and SpanFact-c and the generally very high scores on CNN/DM, which leave less room for improvements compared to the XSum dataset. We also see a lower Edit% on CNNDM, which aligns with Pagnoni et al. (2021) where 27% of BART summmaries on CNN/DM contained a factual inconsistency.
Improving Factual Consistency in Summarization with Compression-Based Post-Editing
In Table 6, we provide model outputs that illustrate the characteristics of the post-editors studied.
Improving Factual Consistency in Summarization with Compression-Based Post-Editing
We show the results of applying post-editors on additional non-BART models in Table 7. We find similar trends as the post-editors applied to other non-pretrained in the main text.
Improving Factual Consistency in Summarization with Compression-Based Post-Editing
Additionally, we trained a post-editor only on sentence compression data; the model produces the compressed sentence (summary) given an uncompressed sentence (summary), with entities to remove marked with special tokens. This model applied to BART XSum summaries provided similar entity precision but resulted in about a 1.2 drop in ROUGE-1 and a five-point drop in entity recall on the validation set, so we did not include this model.
Improving Factual Consistency in Summarization with Compression-Based Post-Editing
In this section, we provide additional details for our model, baselines, and metrics. To encourage retaining essential information, we filter data points from Filippova and Altun (2013) in which the compressed sentences contain less than 75% the number of tokens in the longer sentence. We compared the effect of sentence compression data on downstream post-editor performance, experimenting with higher compression ratios as well as other sentence compression datasets (Clarke and Lapata, 2008), but these models resulted either in lower ROUGE performance or a decrease in entity precision on the validation set. To create training examples for our post-editor, we inserted one, two, and three entities into the references by applying our perturber. The size of the subsets of reference summaries entity precision of 100 on (training, validation) is (52k, 2.9k) for XSum and (160k, 6.5k) for CNN/DM. We then sample 200k data points to train on a dataset of size similar to XSum and found that doubling the size of this data did not give further improvements.
Improving Factual Consistency in Summarization with Compression-Based Post-Editing
We experimented with other approaches for marking extrinsic errors such as model-based approaches Zhou et al. (2021) and Deng et al. (2021), but the entity overlap approach performed better and is more interpretable.
Improving Factual Consistency in Summarization with Compression-Based Post-Editing
We use the following inference parameters: (beam size, min generation length, max generation length, length penalty) for XSum = (6, 11, 62, 1.0) and CNN/DM = (4, 40, 140, 2.0). We retrain BART-large on XSum, and for CNN/DM we run inference from the fairseq/bart-large-cnn 1 checkpoint from the transformers library (Wolf et al., 2020).
Improving Factual Consistency in Summarization with Compression-Based Post-Editing
For ReDRESS, we train BART-large rather than the BART-base model used in the original paper for  A US contractor who was held hostage in Syria has been freed, the US State Department says. The victims were farming in an area... close proximity to Mount Sinabung. The volcano was still spewing ash on Sunday. ... Rescue teams are still scouring the area, looking for more victims who may have been killed or badly burned by the hot gas and ash clouds released in the eruption.
Improving Factual Consistency in Summarization with Compression-Based Post-Editing
Rescue workers in Indonesia are searching for more than 20 people feared dead after the Mount Sinabung volcano erupted on Saturday.
Improving Factual Consistency in Summarization with Compression-Based Post-Editing
Rescue workers in Indonesia are searching for more victims who may have been killed when the Mount Sinabung volcano erupted on Saturday.
Improving Factual Consistency in Summarization with Compression-Based Post-Editing
Rescue workers in Indonesia are searching for more victims who may have been killed when the Mount Sinabung volcano erupted.  consistency with the above models. A key component of this post-editor is the over-generation and reranking of the edited summaries; we rerank according to entity precision over the input. We also vary how entities are matched in the ReDRESS retrieval component and use exact string matching to match how we filtered the clean data subset.
Improving Factual Consistency in Summarization with Compression-Based Post-Editing
For CCGS, we apply the XSum-trained reranker to both XSum and CNN/DM.
Improving Factual Consistency in Summarization with Compression-Based Post-Editing
For DAE, we apply the DAE_xsum_human_best model. 2 We provide our BERTScore run hash. 3
Focus Attention: Promoting Faithfulness and Diversity in Summarization
Document summarization -producing the shorter version of a document while preserving salient information (Mani, 2001;Nenkova and McKeown, 2011) -is challenging even for humans. Today, systems can generate summaries with a high level of fluency and coherence. This is due to recent advances such as sequence-to-sequence architectures (seq2seq) with attention and copy mechanism (Hochreiter and Schmidhuber, 1997;Bahdanau et al., 2015;Gu et al., 2016), fully attention-based Transformer architectures (Vaswani et al., 2017), and large pretrained language models (Devlin et al., A GOLD: Australia has expelled an Israeli diplomat saying Israel was behind the forging of Australian passports linked to the murder of a Hamas operative in Dubai. PEGASUS: Australia has expelled an Israeli diplomat after concluding that forged Australian passports used in the killing of a Hamas militant in Dubai were issued by Israel. Our PEGFAME model: The Australian government has expelled an Israeli diplomat over the use of forged Australian passports in the killing of a Hamas militant in Dubai.
Focus Attention: Promoting Faithfulness and Diversity in Summarization
Israel has summoned the Australian ambassador to complain after the Australian government said forged passports used in the killing of a Hamas operative in Dubai belonged to Netanyahu's foreign ministry. The Australian government has ordered Israel to withdraw an officer over the use of forged Australian passports used by the 2013 murder of a Lebanese opposition figure in Dubai.
Focus Attention: Promoting Faithfulness and Diversity in Summarization
Israel hasracuse withdrawn an envoy after the Australian government said it concluded that Israeli agents used forged passports used to kill a Dubai Bendigo businessman. The Australian government has recalled an Israeli diplomat over accusation that fake Australian passports used 436 kilometres (300 miles) from Canberra in the death of a Hamas militant were stolen by Israeli agents. C Our PEGFAME model with novel Focus Sampling Australia has expelled an Israeli diplomatic staff after accusing the country's security agency, the Israeli military's intelligence agency, of being responsible for the use of Australian visas used in the killing of a Palestinian. The Australian government has expelled an Israeli diplomatic staff after it said the country was responsible for the use of Australian visas used in the killing of a Palestinian in the Middle East.
Focus Attention: Promoting Faithfulness and Diversity in Summarization
Figure 1: Block A shows the best predictions from PEGASUS and our PEGFAME (PEGASUS with FAME) model, along with the GOLD summary for an XSUM article. Block B presents diverse summaries generated from PEGASUS using top-k and nucleus sampling. Block C shows diverse summaries generated using our PEGFAME model with Focus sampling. The text in orange is not supported by the input article. Radford et al., 2018;Yang et al., 2019;Dong et al., 2019a;Song et al., 2019;Lewis et al., 2019;Rothe et al., 2020;Raffel et al., 2019;Zhang et al., 2019).
Focus Attention: Promoting Faithfulness and Diversity in Summarization
However, in terms of summary quality, many challenges remain. For example, generating summaries that are faithful to the input is an unsolved problem (Kryscinski et al., 2020;Maynez et al., 2020;Gabriel et al., 2020). Furthermore, there can be multiple equally good summaries per source docu-ment. Neural generation models fail to account for this and tend to generate outputs with low diversity due to standard likelihood training, approximate decoding objectives, and lack of high quality multireference datasets (Fan et al., 2018;Kulikov et al., 2019;Freitag et al., 2020;Choi et al., 2020). Not much attention has been given to generation of diverse, yet faithful summaries -two goals are often challenging to achieve simultaneously (Hashimoto et al., 2019); a model can produce diverse outputs through sampling (Fan et al., 2018;Holtzman et al., 2020), but at the cost of quality.
Focus Attention: Promoting Faithfulness and Diversity in Summarization
In this paper we introduce a Focus Attention MEchanism (or FAME) to transformer-based seq2seq architectures. FAME is inspired by how humans write summaries. Specifically, FAME aims to perform source-side planning to focus the summary on supported and topical content. FAME achieves this through a novel technique which augments standard contextual representations with a dynamic source-conditioned vocabulary biasing layer. We present the following experimental findings: FAME promotes summaries faithful to the source When evaluated on the BBC extreme summarization task (XSUM;Narayan et al., 2018), experiments with two state-of-the-art summarizers -ROBERTAS2S (Rothe et al., 2020) and PEGA-SUS (Zhang et al., 2019) -show that both models generate summaries that are more faithful to their input documents when augmented with FAME, in comparison with their vanilla counterparts. 1 Faithfulness is measured through a variety of previously proposed metrics. In addition, we leverage the manually annotated document-summary pairs for faithfulness from Maynez et al. (2020) and train a scorer which serves as an efficient proxy for expensive human evaluations. We call this metric BERTFaithful. FAME enables diverse summaries FAME, by design, supports Focus Sampling -a technique that is more effective in sampling topically relevant tokens to generate diverse, yet topically consistent and faithful outputs, than other sampling methods (Fan et al., 2018;Holtzman et al., 2020). Figure 1 illustrates how focus sampling generates better summaries than other sampling methods. We demonstrate the effectiveness of our new Focus Sampling technique using a variety of existing diversity and faithfulness measures. Empirically, we find that optimizing for high diversity often comes at the cost of faithfulness. Thus FAME provides a mechanism for trading-off high faithfulness with better diversity in summarization.
Focus Attention: Promoting Faithfulness and Diversity in Summarization
Diverse Generation Models There has been a surge of interest in making language models generate more diverse and human-like outputs. Vijayakumar et al. (2018) and Kulikov et al. (2019) diversify beam search, using a task-specific scoring function, or constrain beam hypotheses to be sufficiently different. Others avoid text degeneration by truncating the unreliable tail of the probability distribution at each decoding step, either by sampling from the top-k tokens (Top-k Sampling;Fan et al., 2018) or by sampling from a dynamic nucleus of tokens with the bulk of the probability mass (Nucleus Sampling;Holtzman et al., 2020). Others modify the training objective to make the distribution sparse (Martins et al., 2020) or assign lower probability to unlikely generations (Welleck et al., 2019a).
Focus Attention: Promoting Faithfulness and Diversity in Summarization
For conditional text generation, most work focuses on generating diverse questions (Narayan et al., 2016;Dong et al., 2017;Sultan et al., 2020;Wang et al., 2020b) or paraphrases (Li et al., 2016b;Dai et al., 2017;Xu et al., 2018;Cao and Wan, 2020). Following Gehrmann et al. (2018), Cho et al. (2019 use a mixture of experts to sample different binary masks on the source sequence for diverse content selection for summarization. Our focus sampling is similar to top-k and nucleus sampling methods; in that it truncates the tail of the probability distribution. However, instead of truncating it at each decoding step, it biases the decoder proactively to generate output from a set of tokens which are topically-relevant to the input.
Focus Attention: Promoting Faithfulness and Diversity in Summarization
Given an input document X 1:n , we aim to generate its summary Y 1:m , where n and m are input and output sequence lengths. We address this prob-  Figure 2: A Transformer-based encoder-decoder architecture with FAME. lem using seq2seq architectures with Transformer encoder and decoder, augmented with FAME, as depicted in Figure 2. FAME learns a distribution t x i for each input token x i over the vocabulary, measuring similarity of x i (in context) to the tokens in the vocabulary. The vocabulary distributions, t x i , for all x i are combined to form a dynamic vocabulary bias that is added to the decoder logits. This mechanism enhances the conditioning on the input source and encourages the decoder to generate tokens that are topically similar to the input.
Focus Attention: Promoting Faithfulness and Diversity in Summarization
Transformer-based seq2seq Model The encoder uses BERT Transformer layers with multiheaded self-attention to encode X to a vector sequence X = x 1 , . . . , x n , with x i ∈ R h , where h is the size of hidden representation. The decoder uses an identical architecture, except that at decoding step t, layer l adds a conditional representation y l t ∈ R h for the token y t by attending to the output representation Y l−1 1:t−1 = y l−1 1 , . . . , y l−1 t−1 generated so far through self-attention and by attending to the input contextual representation X through encoderdecoder attention. The probability of predicting the next token y t from a vocabulary V is: by minimizing cross-entropy at each decoding step:
Focus Attention: Promoting Faithfulness and Diversity in Summarization
where,Ŷ 1:m is the human-written summary.
Focus Attention: Promoting Faithfulness and Diversity in Summarization
Focus Attention MEchansim (FAME) It is challenging for a decoder to obtain all relevant information from the conditional representation y L t to learn the vocabulary output logits such that predictions y t are consistent with the input. Other modeling factors, specifically the decoder language model, can overwhelm model predictions. FAME (Figure 2) addresses this by introducing a short-circuit from the source to the vocabulary output logits via a source-conditioned bias on vocabulary items.
Focus Attention: Promoting Faithfulness and Diversity in Summarization
We take the encoder representation X = x 1 , . . . , x n and learn a Token-level Vocabulary Distribution t x i = gelu(x i W 1 )W 2 E ∈ R |V | , for each token x i in the input sequence X. t x i measures the contextual similarity of the input token x i to the tokens in the vocabulary; W 1 ∈ R h×h and W 2 ∈ R h ×h are parameters of newly introduced dense layers, h is the intermediate filter size. We define a Source-conditioned Vocabulary Distribution as t X = 1/n n i=1 t x i ∈ R |V | as an average of token-level vocabulary distributions for tokens present in the input sequence X, capturing the similarity of X to the tokens in the vocabulary.
Focus Attention: Promoting Faithfulness and Diversity in Summarization
Let a L t ∈ R n be the encoder-decoder attention distribution over the source tokens for the output token y t and the final decoder layer L. We use a L t to produce a weighted sum of the token-level vocabulary distributions to compute a dynamic vocabulary bias, or Focus Bias f t = n i=1 a L t,i t x i ∈ R |V | at decoding step t. We modify the probability of predicting the next token y t from a vocabulary V as:
Focus Attention: Promoting Faithfulness and Diversity in Summarization
We call this Focused Probability Distribution, and it modifies the output logits dynamically to put more focus on those tokens in the vocabulary which are similar to the attended tokens in X. The focus bias introduces a human-inspired control to the model where we do not generate the output in a fully abstractive manner (as in Eq. (1)), but we proactively generate output tokens that are similar to the input tokens (as in Eq.
Focus Attention: Promoting Faithfulness and Diversity in Summarization
(2)).
Focus Attention: Promoting Faithfulness and Diversity in Summarization
We aim to guide our focus bias f t to be a better representative of the topical content relevant for the task. We achieve this by using the human-written summaryŶ as a proxy for the topical content of the input and impose the following prior on the source-conditioned vocabulary distribution t X :
Focus Attention: Promoting Faithfulness and Diversity in Summarization
We further refine Eq. (3) by replacingŶ withŶ c = Y −F , where F is a set of |F | most frequent tokens in the vocabulary, 2 to improve focus on content words. Our final loss function is then
Focus Attention: Promoting Faithfulness and Diversity in Summarization
where, λ is an hyper parameter. 3 By enforcing t X to be a topic distribution for the input X, we encourage the focus bias f t to promote topically relevant tokens, and subsequently generate topically consistent outputs. Importantly, our focus bias with target-induced topic distribution is task-agnostic and less vulnerable to reference divergence issues (Dhingra et al., 2019;Maynez et al., 2020), and can learn any property embodied in the target relevant for the task. For example, depending on the task, f t can learn to favour input tokens (e.g., for mostly extractive summaries) or new tokens (e.g., for mostly abstractive summaries). This is in sharp contrast to models that introduce task-specific priors, e.g., the pointer-generator network (See et al., 2017) that can copy words from the source text, but does not do well on extreme summarization which is highly abstractive in nature (Narayan et al., 2018).
Focus Attention: Promoting Faithfulness and Diversity in Summarization
Focus Sampling: Promoting Diversity in Faithful Generation We introduce Focus Sampling with FAME to construct a subset V k ⊆ V by sampling k tokens from the topic distribution t X (Focus sample,k ). Then, we modify Eq. (2) as
Focus Attention: Promoting Faithfulness and Diversity in Summarization
For document summarization, the subset V k will capture topically salient tokens necessary to generate a summary; F is always added to V k to ensure that the model has access to function words. By tuning the parameters of sampling, we can enforce the model to control the faithfulness or diversity of the outputs. Focus sampling has similarities to top-k (Div top,k ; Fan et al., 2018) and nucleus sampling (Div nucleus ; Holtzman et al., 2020); in that they all aim to promote diversity. At each decoding step, the top-k sampling diversifies the generation process by sampling a token from the top k tokens in the final output distribution. Similarly, nucleus sampling samples from a dynamic nucleus of tokens containing the vast majority (with a cumulative probability p) of the probability distribution. Both top-k and nucleus sampling shorten the tail of the output distribution at each decoding step, whereas focus sampling constrains the decoder to use a fixed and topically relevant vocabulary V k . Unlike the other two techniques, Focus sample,k can also benefit from standard beam search decoding, leading to superior generation that is not only diverse, but also consistent with the input document.
Focus Attention: Promoting Faithfulness and Diversity in Summarization
In this section we present our experimental setup to assess the ability of our FAME models to generate faithful summaries and to demonstrate that focus sampling is more effective in generating diverse and faithful summaries than other sampling-based decoding methods.
Focus Attention: Promoting Faithfulness and Diversity in Summarization
We evaluate FAME models on extreme document summarization (XSUM; Narayan et al., 2018). The XSUM summaries, are extreme in that the documents are summarized into single-sentence summaries. These summaries demonstrate a high level of abstractiveness, and generating them automatically requires document-level inference, abstraction, and paraphrasing. Due to their extreme nature, XSUM summaries are ideal to evaluate FAME models' ability to capture the theme of the document. 4 We use on the original cased version consisting of 204,045/11,332/11,334 training/validation/test document-summary pairs. During training, the input documents are truncated to 512 tokens. The 4 We further experiment with long-form story highlight generation (CNN/DM;Hermann et al., 2015) and two text editing tasks: Sentence Fusion (Geva et al., 2019) and Sentence Splitting (Botha et al., 2018). Their results can be found in Appendix B and C. Our FAME models achieve SOTA on both text-editing tasks. length of the summaries are limited to 64.
Focus Attention: Promoting Faithfulness and Diversity in Summarization
We introduce FAME to two popular seq2seq architectures: RoBERTa initialized seq2seq (ROBERTAS2S, Rothe et al., 2020) and PEGASUS (Zhang et al., 2019). We refer ROBERTAS2S models with FAME as ROBFAME and PEGASUS with FAME with PEGFAME.
Focus Attention: Promoting Faithfulness and Diversity in Summarization
We experiment with ROBERTAS2S-Large with shared encoder and decoder; it has 24 layers, a hidden size of 1024, filter size of 4096, 16 attention heads, and a vocabulary with 50K sentence pieces (Kudo and Richardson, 2018). ROBERTAS2S has around 455M parameters and ROBFAME has an additional 8M parameters.
Focus Attention: Promoting Faithfulness and Diversity in Summarization
The best-performing PEGASUS model from Zhang et al. (2019) is not directly comparable with ROBERTAS2S. It does not share the encoder and decoder, it only has 16 layers, a hidden size of 1024, filter size of 4096, 16 attention heads, with a total of 568M parameters, and it also uses a much larger vocabulary with 91K sentence pieces. Hence, we trained our own PEGASUS model. We use the same architecture as ROBERTAS2S and pretrain it on a mixture of C4 (Raffel et al., 2019) and Huge-News (Zhang et al., 2019) datasets with the original objective of generating salient GAP-sentences.
Focus Attention: Promoting Faithfulness and Diversity in Summarization
Our experiments focus on this newly trained PEGASUS model which has same number of parameters and vocabulary as ROBERTAS2S. But in contrast to ROBERTAS2S, the encoder-decoder attention in PEGASUS is pretrained. This allows us to analyse how focus attention affects pretrained (PEGASUS) vs randomly-initialized (ROBERTAS2S) encoder-decoder attentions. 5
Focus Attention: Promoting Faithfulness and Diversity in Summarization
Lexical Overlap We report ROUGE F1 scores (Lin and Hovy, 2003) against reference summaries; in particular, we report on ROUGE-1 and ROUGE-2 for informativeness and ROUGE-L for fluency. 6 Semantic Similarity We report BERTScore (Zhang et al., 2020) which computes the contextual similarity between a candidate and its reference summary. Faithfulness ROUGE and BERTScore do not correlate well with faithfulness of the generated summaries (Maynez et al., 2020). Human evaluation is traditionally considered as the gold standard for measuring faithfulness. But recent research has shown that even human evaluation has shortcomings (Schoch et al., 2020). Moreover, it is prohibitively expensive. This has led to the proposal of meta-evaluation metrics for various generation tasks (Durmus et al., 2020;Kryściński et al., 2019;Sellam et al., 2020;Rei et al., 2020). We evaluate FAME models on semantic inference metrics such as textual entailment (Pasunuru and Bansal, 2018;Welleck et al., 2019b;Falke et al., 2019;Kryscinski et al., 2019) and question answering (Arumae and Liu, 2019; Wang et al., 2020a). In particular, we report the probability of a summary entailing (ent.) its input document (Maynez et al., 2020) and QA-based Feqa scores (Durmus et al., 2020). For ent. scores, we train an entailment classifier by fine-tuning a BERT-Large pretrained model (Devlin et al., 2019) on the Multi-NLI dataset (Williams et al., 2018). For Feqa, we use a fine-tuned BART (Lewis et al., 2019) language model for question generation to generate questions from the summaries, and a BERTbase model fine-tuned on SQuAD (Rajpurkar et al., 2018) to answer the generated questions with input document as context. 7 In addition to ent. and Feqa, we train a scorer leveraging manually annotated document-summary pairs for faithfulness, as a surrogate for human evaluation and call this metric BERTFaithful. 8 In particular, we finetune a BERT-Base classi- 7 We used the Feqa code available here: https:// github.com/esdurmus/feqa/. 8 A very similar scorer was used in the GEM benchmark (Gehrmann et al., 2021) to identify and extract the subset with faithful reference summaries from the XSum dataset (Narayan et al., 2018). fier on 500 manually annotated document and gold summary pairs for the XSum dataset from Maynez et al. (2020) to predict whether a summary is faithful to the input document or not. 9 We report the percentage of summaries that were faithful ( 1 N i 1[p i (faithful) > 0.5]) and the model's confidence to generate faithful summaries ( 1 N i p i (faithful)); N is the total number of examples in the test set.
Focus Attention: Promoting Faithfulness and Diversity in Summarization
Diversity We report the number of times (out of n), a model is able to generate a completely new summary (Unique), and Distinct-N (Li et al., 2016a), measuring the lexical diversity in the generated summaries. Distinct-N is estimated as the number of distinct n-grams of order n divided by the total number of n-grams of the same order, in all generated summaries.
Focus Attention: Promoting Faithfulness and Diversity in Summarization
Finally, we also report the average length of summaries (Len.), repetition errors (Rep., estimated as the percentage of summaries with at least one repetition of rare or content words), and ROUGE-1 precision against the input document (R1, P%), to better understand their quality.
Focus Attention: Promoting Faithfulness and Diversity in Summarization
5 Results FAME Summaries are More Fluent, Informative and Faithful. Table 1 presents results comparing our FAME models, ROBFAME and PEG-FAME, against their counterparts ROBERTAS2S 9 Out of 500, 90% of the document-summary pairs were used for training and the rest 50 document-summary pairs were used for validation. We used the validation set to estimate Spearman's correlation coefficients of different metrics with the human assessment for faithfulness. We found that both entailment scores (ent.) and BERTFaithful are moderately correlated with faithfulness with correlation coefficients of 0.4387 and 0.3889, respectively. As such, we believe that BERTFaithful works as an efficient proxy for expensive human evaluation for faithfulness for XSum summaries. More work is needed to understand if BERTFaithful generalizes to other datasets.
Focus Attention: Promoting Faithfulness and Diversity in Summarization
Dist.  and PEGASUS, respectively. Both FAME models clearly outperform their vanilla counterparts in terms of generating summaries that are more fluent (see RL and Rep.), more informative (see R1, R2 and BERTSc.) and more faithful (see ent., Feqa and BERTFaithful). Among all four models, PEGFAME summaries are most fluent, informative and faithful. We further did pairwise comparisons for all measures in Table 1 and found that all differences are statistically significant except for BERTScore and faithfulness measures between PEGASUS and PEGFAME. 10 These assessments demonstrate that FAME models aid both ROBERTAS2S and PEGA-SUS in generating fluent, faithful and relevant summaries, but are more effective in ROBERTAS2S than in PEGASUS for extreme summarization.
Focus Attention: Promoting Faithfulness and Diversity in Summarization
Generating Diverse and Faithful Summaries with Focus Sampling. Table 2 presents results assessing focus sampling (Focus sample,k ), top-k sampling (Div top,k ) and nucleus sampling (Div nucleus ), for their abilities to generate diverse and faithful summaries. For Focus sample,k , we choose k = 10, 000. We follow Holtzman et al.
Focus Attention: Promoting Faithfulness and Diversity in Summarization
(2020) and choose k = 640 and the nucleus probability p = 0.95, for Div top,k and Div nucleus , respectively. For Focus sample,k , we decode with a beam size of 4. We also report Focus sample,k with Div top,k and Div nucleus to assess if they can benefit one-another. In each setting we sample 10 sum-10 All significance tests in this work are pairwise comparisons (one-way ANOVA with posthoc Tukey HSD tests; p < 0.01). maries for each input document. For all metrics, we report the average over all 10 samples. 11 Both Div top,k and Div nucleus almost always generate a new summary. In comparison Focus sample,k generates 1.61 and 2.77 unique summaries using ROBFAME and PEGFAME models, respectively. Div nucleus tends to generate the most distinct unigrams, bigrams, and trigrams. Interestingly, Focus sample,k summaries have a more diverse collection of unigrams than in Div top,k summaries (3.5% vs 2.3% for ROBFAME and 2.4% vs 1.9% for PEGFAME).
Focus Attention: Promoting Faithfulness and Diversity in Summarization
The high diversity in Div top,k and Div nucleus comes at the cost of faithfulness; summaries generated with these sampling techniques have poor entailment scores. Focus sample,k , on the other hand, generates summaries which entail documents the most. It also has the highest ROUGE scores across the board. Some of the generated examples can be seen in Figure 1. More predictions from other models can be found in Appendix E. Augmenting Div top,k and Div nucleus with Focus sample,k is not desirable because, though it increases diversity in terms of uniqueness and Distinct-3 scores, faithfulness suffers again.
Focus Attention: Promoting Faithfulness and Diversity in Summarization
Comparing results in Table 2 to the results in Table 1, it is clear that diversity comes at the cost of quality (e.g., RL/ent. scores for ROBFAME and ROBFAME-Focus sample,k are 34.81/41.3 and 31.0/34.3, respectively). However, Focus sample,k is superior to both Div top,k and Div nucleus in gen-6085 erating better quality summaries.  Focus Attention and Sampling Work Differently in ROBFAME and PEGFAME. Since both encoder-decoder and focus attention parameters of ROBFAME are randomly initialized, they learn to compliment each other and learn a peaky topic distribution. On the other hand, since PEGFAME's encoder-decoder attention is pre-trained, there is a push-pull effect between it and focus attention. This results in a smoother topic distribution, as seen in Figure 3. 12 Although we see that both models' token sets capture the target intent well, the peaky distribu-12 This difference in topic distributions is consistent across the whole test set. We compute the peakiness score of a topic distribution as the slope of the line connecting logits of the top-1st token to the top-100th token. The average peakiness scores across the XSUM testset for ROBFAME and PEGFAME are 1.25 (51 • ) and 0.45 (24.3 • ), respectively.  tion of ROBFAME enables more accurate predictions than that of PEGFAME, in a controlled generation setting. A comparison is presented in Figure 4 where we show how ROUGE-1 scores vary when we use only top-k tokens from t X for generation. 13 We observe that ROBFAME consistently outperforms PEGFAME with the lower values of k ∈ {50, 100, 200, 500, 1000}. Further, we observe that ROBFAME generates fewer unique summaries (1.61 vs 2.77) but has higher Distinct-N scores (3.5/22.4/43.9 vs 2.4/16.5/34.2) than PEGFAME, with Focus sample,k in Table 2. This can be again be attributed to how FAME works differently in ROBFAME and PEG-FAME. When V k is sampled from ROBFAME's peaky distribution, the beam search decoding often tends to generate similar summaries (leading to a lower Uniqueness score) as the sampled V k s do not diverge by much from each other. But when it does diverge, the decoder tends to generate completely new summaries (leading to higher Distinct-N scores).
Focus Attention: Promoting Faithfulness and Diversity in Summarization
Currently, we set k = 10, 000 for our focus sampling experiments following our observations in Figure 4. Future work will focus on how to better leverage trade-off between diversity and faithfulness by controlling the peakiness of the topic distribution t X .
Focus Attention: Promoting Faithfulness and Diversity in Summarization
Ablations and SOTA Comparisons We emphasize that FAME or focus sampling does not aim to improve on state-of-the-results in terms of ROUGE, but to generate more faithful or diverse summaries while maintaining their quality. For completeness, we compare our ROBFAME and PEGFAME models to their ablations and other state-of-the-art models on XSUM in Table 3.
Focus Attention: Promoting Faithfulness and Diversity in Summarization
We report ROUGE scores for FAME in the ideal scenario (ORACLE) where it focuses on all the correct tokens in the input, i.e., the topic distribution t X is identical to the distribution observed in the reference summary. These models generate summaries with very high ROUGE scores when the model is given the correct tokens to focus on. The gap between the ORACLE and FAME scores suggests that there is still a lot of work to be done in this space. Focus attention without any topical supervision (models w/o Eq. (3)) is not significantly better than the baselines. But ROBFAME and PEG-FAME (trained with joint supervision in Eq. (4)) significantly outperform ROBERTAS2S and PEGA-SUS, respectively.
Focus Attention: Promoting Faithfulness and Diversity in Summarization
Our best model PEGFAME performs better than PtGen (See et al., 2017) (Zhang et al., 2019). This can be expected as the number of parameters in PEGFAME is far less than that in the original PEGASUS.
Focus Attention: Promoting Faithfulness and Diversity in Summarization
We introduced FAME, a new attention mechanism which dynamically biases the decoder to proactively generate tokens that are topically similar to the input. FAME enhances the faithfulness of existing state-of-the-art abstract summarization models while improving their overall ROUGE scores. Finally, our newly introduced focus sampling technique is a better alternative to top-k or nucleus sampling to generate diverse set of faithful summaries.
Focus Attention: Promoting Faithfulness and Diversity in Summarization
We thank Sebastian Gehrmann, Slav Petrov, the reviewers, and the action editor for their invaluable feedback.
Focus Attention: Promoting Faithfulness and Diversity in Summarization
The nature of text generation leads to multiple ethical considerations when applied to applications. The main failure mode is that the model can learn to mimic target properties in the training data that are not desirable.
Focus Attention: Promoting Faithfulness and Diversity in Summarization
Since models create new text, there is the danger that they may neither be faithful to the source material nor factual. This can be exacerbated when the data itself has highly abstractive targets, which require the model to generate words not seen in the source material during training. This often leads the model to generate content inconsistent with the source material (Kryscinski et al., 2020;Maynez et al., 2020;Gabriel et al., 2020).
Focus Attention: Promoting Faithfulness and Diversity in Summarization
Trustworthy Data If the data itself is not trustworthy (comes from suspect or malicious sources) the model itself will naturally become untrustworthy as it will ultimately learn the language and topics of the training data. For instance, if the training data is about Obama birther conspiracies, and the model is asked to generate information about the early life of Obama, there is a risk that such false claims will be predicted by the model.
Focus Attention: Promoting Faithfulness and Diversity in Summarization
Similarly, biases in the data around gender, race, etc., risk being propagated in the model predictions, which is common for most NLP tasks. This is especially true when the models are trained from non-contemporary data that do not represent current norms and practices (Blodgett et al., 2020).
Focus Attention: Promoting Faithfulness and Diversity in Summarization
The above considerations are non-malicious, in that the model is merely learning to behave as its underlying source material. If users of such models are not aware of these issues and do not account for them, e.g., with better data selection, evaluation, etc., then the generated text can be damaging.
Focus Attention: Promoting Faithfulness and Diversity in Summarization
Generation models can also be misused in malicious ways. These include generating fake news, spam, and other text meant to mislead large parts of the general population. 
Focus Attention: Promoting Faithfulness and Diversity in Summarization
Following Rothe et al. (2020), the encoder and decoder of ROBERTAS2S and ROBFAME models are initialized with public RoBERTa checkpoints. The encoder and decoder parameters are shared in both cases. Only the encoder-decoder attention parameters are initialized randomly. For ROBFAME, the focus attention parameters are also randomly initialized. We experiment with large RoBERTa checkpoints with 24 layers, a hidden size of 1024, filter size of 4096, 16 attention heads, and a vocabulary with 50K sentence pieces (Kudo and Richardson, 2018). ROBERTAS2S has around 455M parameters and ROBFAME has 463M parameters, with an additional 8M parameters. Our PEGASUS and PEGFAME implementation also have the same configuration, except for the encoder-decoder attention parameters which are pretrained. We used Cloud TPU v3 accelerators for training. All models are fine-tuned on the target task using Adam with a learning rate of 0.05. We use a linear learning rate warm up with 40k steps, normalized by the square root of the hidden size, and a square root decay. We do not perform any tuning on these hyperparameters. We use a global batch size of 128 document-summary pairs. We adapt to different number of training steps depending on the training data sizes. Models are trained for 400k and 200k steps for CNN/DM and XSUM respectively, saving check-points every 1000 steps. We choose the best model based on ROUGE-L performance on the respective validation set.
Focus Attention: Promoting Faithfulness and Diversity in Summarization
The vocabulary for functional tokens F is constructed by taking the most frequent sentence pieces in the training set. We tune |F | using the respective validation sets; for XSUM, we choose f = 500 frequent sentence pieces and for CNN/DM, f = 1000. For all our experiments with the FAME models, the beam size is set to 4.
Focus Attention: Promoting Faithfulness and Diversity in Summarization
We use Cloud TPU v3 accelerators for computing entailment scores which takes about 20 minutes for the two datasets' test sets. Question generation and answering for Feqa are run on a NVIDIA V100 GPU, and it takes between 8-12 hours for one setting of each test set.
Focus Attention: Promoting Faithfulness and Diversity in Summarization
The CNN/DM dataset (Hermann et al., 2015) consists of 287,227/13,368/11,490 train-   Table 4 and 5 present complete results for CNN/DM dataset. We see similar kind of improvements as observed in Table 1, except for ROUGE-2 for ROBFAME which is 0.23 points worse than the ROBERTAS2S baseline. Our best model PEG-FAME performs better than both copy mechanism models: LSTM-based PtGen (See et al., 2017) and Transformer-based SAGCopy (Xu et al., 2020). PEGFAME performs worse when compared with T5 (Raffel et al., 2019), the original PEGASUS (Zhang et al., 2019) and ProphetNet (Qi et al., 2020). This can be expected as the number of parameters in PEGFAME is almost half of T5 or ProphetNet, and is 100M less than that in the original PEGASUS. ROBFAME performs worse than ROBERTAS2S on both ent. and Feqa measures for CNN/DM, similar to ROUGE-2 in Table 4. We hypothesize that this is due to the extractive nature of the CNN/DM dataset and the fact that it is not able to copy to-  kens from the input to the necessary extent as the encoder-decoder attention is not pre-trained. Moreover, Feqa scores for ROBERTAS2S and ROBFAME may not be fully comparable due to variation in their summary lengths and the number of Feqa questions generated; the ROBFAME summaries, on average, are 3 words longer and generate 1.2 more questions than that of ROBERTAS2S. Nevertheless, we don't see this kind of drop in ¬cont. scores (i.e., summary not contradicting, either entailed by or neutral to the document) and BERTScores.
Focus Attention: Promoting Faithfulness and Diversity in Summarization
We also train the FAME models on two text editing tasks: (i) for sentence fusion -the problem of combining multiple sentences into a single coherent sentence -we used the "balanced Wikipedia" portion of the DiscoFuse dataset (Geva et al., 2019), and (ii) for split-and-rephrase -the reverse task of sentence fusion -we used the WikiSplit dataset (Botha et al., 2018), which consists of 1M examples of sentence splits extracted from the Wikipedia edit history. As the name suggests, both text editing tasks require a low degree of abstraction. For both the tasks, we train the models for 300k steps with a global batch size of 256. The input and output are padded to a length of 128, which covers 100% of the training, evaluation and test data. The vocabulary for functional tokens F is constructed by taking the top 100 and 500 sentence pieces for DiscoFuse and WikiSplit respectively.
Focus Attention: Promoting Faithfulness and Diversity in Summarization
We report corpus-level BLEU 14 , the exact match accuracy, and SARI scores (Xu et al., 2016) 15 . The results can be seen in Table 6. The vanilla PEGA-SUS model already beats the current state-of-the-art on both DiscoFuse and WikiSplit. The PEGFAME 14 We use NLTK v3.2.2 with case sensitive scoring to estimate BLEU scores. 15 SARI is a lexical similarity metric which compares the model's output to multiple references and the input in order to assess the model's ability to add, delete, and keep an n-gram.
Focus Attention: Promoting Faithfulness and Diversity in Summarization
It's implementation is available at: https://github.com/tensorflow/ tensor2tensor/blob/master/tensor2tensor/ utils/sari_hook.py.  D Controlled Generation with focus attention using Top-k tokens Table 7 presents results from our controlled summary generation experiments with top-k tokens from t X using focus attention (Focus top,k ) on the XSUM test set. In Figures 3 and 4, we describe how ROBFAME consistently outperforms PEGFAME at lower values of k ∈ {50, 100, 200, 500, 1000} due to their peaky and smooth t X , respectively. While Figure 4 only plots ROUGE-1 F1 scores, Table 7 additionally reports ROUGE-2, ROUGE-L, entailment, Feqa, and BERTScores. Figure 6 presents predictions from models using Focus top,k for the article presented in Figures 1 and 5 Figure 5. The predictions from Div top,k and Div nucleus are omitted due to the prescribed limit on the number of pages allowed for the Appendix. Please find them on the arXiv version at https://arxiv.org/ abs/2105.11921. We experiment with limiting FAME models to different sizes of vocabulary V k using the topic distribution t X ; in particular, we experiment with k = {50, 100, 200, 500, 1000, 10000}. We also report numbers for ROBERTAS2S, ROBFAME, PEGASUS and PEGFAME, using the whole vocabulary of size 50k. The bold results in each block are the best performing ROBERTAS2S-based and PEGASUS-based models.
Focus Attention: Promoting Faithfulness and Diversity in Summarization
Australia has expelled an Israeli diplomat saying Israel was behind the forging of Australian passports linked to the murder of a Hamas operative in Dubai.
Focus Attention: Promoting Faithfulness and Diversity in Summarization
Australia's foreign minister said these were "not the actions of a friend". The UK took similar action in March, after concluding that Israel was responsible for the use of forged UK passports in the plot. The Israeli foreign ministry said Australia's decision was disappointing. Ministry spokesman Yigal Palmor said it was "not in line with the importance and the quality of the relationship between our countries". 'Sorrow not anger' At least four forged Australian passports were used in the killing of Mahmoud al-Mabhouh in Dubai in January. The originals belonged to Australians living in Israel. The Australian government said a police investigation had left it in no doubt that the Israeli authorities were behind "the abuse and counterfeiting of the passports". As a result Foreign Minister Stephen Smith asked Israel to withdraw a diplomat, whom he did not identify. "The decision to ask Israel to remove from Australia one of its officers at the Israeli embassy in Canberra is not something which fills the Australian government with any joy," he said. "On the contrary, the decision was made much more in sorrow than in anger." Passports from France, Ireland, Germany and Britain were used in the operation, and in March, the British government expelled an Israeli diplomat from London. The Israeli government has said there is no proof that it was behind the killing, although Dubai officials have said they are 99.9% sure that agents from Mossad were responsible.
Focus Attention: Promoting Faithfulness and Diversity in Summarization
ROBERTAS2S Australia has asked Australia to withdraw an Israeli diplomat from its embassy in Canberra after an alleged plot to kill a Abu Dhabi militant in Dubai.
Focus Attention: Promoting Faithfulness and Diversity in Summarization
Australia has asked Israel to withdraw one of its diplomats from its embassy in Canberra after it admitted it used forged passports.
Focus Attention: Promoting Faithfulness and Diversity in Summarization
Australia has expelled an Israeli diplomat after concluding that forged Australian passports used in the killing of a Hamas militant in Dubai were issued by Israel. PEGFAME
Focus Attention: Promoting Faithfulness and Diversity in Summarization
The Australian government has expelled an Israeli diplomat over the use of forged Australian passports in the killing of a Hamas militant in Dubai. Figure 5: A 2010 BBC article from the XSUM testset, its human written summary and model predictions from ROBERTAS2S, and PEGASUS, with and without FAME. The text in orange is not supported by the input article. ROBFAME (Focus top,k=50 ) Australia has said it will not be expelled an ambassador from Australia following the alleged s agent for the so-called Arab Arab State. ROBFAME (Focus top,k=100 ) Australia has said it will not be expelled an ambassador from Australia following the killing of a terror agent in the Arab world. ROBFAME (Focus top,k=200 ) Australia has said it will not be expelled an ambassador from Australia following the killing of an Australian terror suspect in the Arab world. ROBFAME (Focus top,k=500 )
Focus Attention: Promoting Faithfulness and Diversity in Summarization
Australia has asked Israel to end its diplomatic investigation into an alleged plot to murder an Australian terror suspect. ROBFAME (Focus top,k=1000 )
Focus Attention: Promoting Faithfulness and Diversity in Summarization
Australia has asked Israel to strip an ambassador from its embassy following the death of an Arab man in Dubai. ROBFAME (Focus top,k=10000 ) Australia has asked Israel to withdraw one of its diplomats from its embassy in Canberra following the death of a terror suspect.
Focus Attention: Promoting Faithfulness and Diversity in Summarization
The Israeli government has been expelled from the country after it was found that the country's security agency, the Israeli intelligence agency, was to be to be found to have used a number of the country's out-of-country p when it was used in the Emirates car-j best. PEGFAME (Focus top,k=100 )
Focus Attention: Promoting Faithfulness and Diversity in Summarization
The Israeli government has been expelled from the country after it was found that the country's security agency, the Israeli intelligence agency, had used the country's visas in the Emirates terror. PEGFAME (Focus top,k=200 )
Focus Attention: Promoting Faithfulness and Diversity in Summarization
The Australian government has expelled an Israeli diplomats after it found that the country's security agency, the Israeli intelligence agency, had used the country's visas in the Emirates terror attack. PEGFAME (Focus top,k=500 )
Focus Attention: Promoting Faithfulness and Diversity in Summarization
The Australian government has expelled an Israeli diplomatic staff after accusing the country's security agency, the Israeli intelligence agency, of using a number of Australian visas in the Emirates terror attack. PEGFAME (Focus top,k=1000 )
Focus Attention: Promoting Faithfulness and Diversity in Summarization
Australia has expelled an Israeli diplomatic staff after accusing the country's security agency, the Israeli military's intelligence agency, of being responsible for the use of Australian visas used in the killing of a Palestinian. PEGFAME (Focus top,k=10000 ) Australia has expelled an Israeli diplomat over the use of forged Australian passports in the killing of a Hamas militant in Dubai. Figure 6: Model predictions with focus sampling Focus top,k , a controlled generation setting. The text in orange is not supported by the input article. We note that with smaller values of k, both ROBERTAS2S-based and PEGASUSbased models tend to hallucinate more often. ROBFAME (Focus sample,k ) Australia has asked Israel to strip one of its diplomats from its embassy following the death of an Arab man in Dubai. Australia has asked Israel to end its diplomatic investigation into an alleged plot to murder an Australian terror suspect. Australia has asked Israel to strip one of its diplomats from its embassy in Australia over the death of a terror suspect.
Focus Attention: Promoting Faithfulness and Diversity in Summarization
The Australian government has expelled an Israeli diplomatic staff after accusing it of using a number of Australian visas in the killing of a Palestinian in a car bombing. The Australian government has expelled an Israeli diplomatic staff after it said the country was responsible for the use of Australian visas used in the killing of a Palestinian in a car bombing. Australia has expelled an Israeli diplomatic staff after accusing the country's security agency, the Israeli military's intelligence agency, of being responsible for the use of Australian visas used in the killing of a Palestinian. Australia has expelled an Israeli diplomatic mission after accusing the country's security agency, the Israeli military's intelligence agency, of being responsible for the use of Australian visas used in the killing of a Palestinian in the Arab city of Emirates. The Australian government has expelled an Israeli diplomatic staff after it said the country was responsible for the use of Australian visas used in the killing of a Palestinian in the Middle East.
Focus Attention: Promoting Faithfulness and Diversity in Summarization
Figure 7: FAME model predictions with Focus sample,k (k = 10000). The text in orange is not supported by the input article.
Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling
While modern summarization models generate highly fluent summaries that appear realistic Zhang et al., 2020), these models are prone to generating non-factual and sometimes entirely fabricated content (Cao et al., 2018;Goodrich et al., 2019;Maynez et al., 2020). With the increasing adoption of language generation tools in user-facing products, such unreliability poses severe risks, including the spread of misinformation, panic and other potentially harmful effects (Ranade et al., 2021;Hutson et al., 2021).
Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling
Scientists believe a vaccine for Ebola might not be ready this year. The first vaccine for Ebola took 5 years to be produced by the CBP. Figure 1: Model generated summaries often produce content which is factually inconsistent w.r.t. to the source. FACTEDIT rewrites these summaries by maintaining the abstractiveness but correcting factual errors. Dreyer et al., 2021), a popular approach to fix the factual inconsistencies is via post-editing generated summaries (Cao et al., 2020;. This allows summarization models to focus on fluency and content-relevance while improving factual consistency. However, there is no suitable data for training post-editing models to directly "translate" an incorrect summary to a correct one. Prior work constructed synthetic training data by introducing simple heuristic errors like replacing entities or numbers in reference summaries (Cao et al., 2020), but it is not clear whether such synthetic errors have sufficient coverage and accurately represent the types and distribution of actual errors made by language models. Further, with increasing language generation capabilities, models make more complex factual errors involving discourse structures and paraphrasing which cannot be easily captured with heuristics (Pagnoni et al., 2021). The goal of our work is to develop post-editing models  Figure 2: Architecture framework for FACTEDIT. Using masked versions of existing reference summaries, we use an infilling language model to produce alternative candidates for the mask position. We construct factually incorrect summaries by replacing the mask with the lower ranked candidates. Finally, we train a sequence-to-sequence model for fact correction using the synthetically constructed data.
Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling
We evaluate FACTEDIT on two datasets -CNN/DailyMail (Hermann et al., 2015) and XSum (Narayan et al., 2018) and across nine summarization models with the FRANK benchmark (Pagnoni et al., 2021) for evaluating various categories of factual errors in generated summaries ( §3). The two summarization datasets represent varied distributions of factual errors in models trained on them and hence constitute a good test bed to evaluate the generalizability of our model. We show that FACTEDIT substantially improves factuality scores across two metrics -Ent-DAE (Goyal and Durrett, 2021) and FactCC (Kryscinski et al., 2020). On the Ent-DAE metric, FACTEDIT improves results by ∼11 points (CNN/DM) and ∼31 points (XSum), and on the FactCC metric we show improvements of ∼6 points (CNN/DM) and ∼24 (XSum) points on average across models ( §4). Further, our analysis shows that FACTEDIT effectively corrects diverse error categories without the need for special heuristics or annotations ( §5). An important application of FACTEDIT is to audit summarization systems and facilitate their reliability.
Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling
Our data generation process leverages infilling language models (Donahue et al., 2020) to produce candidates to fill masked phrases in a summary sentence. We mask parts of the input and use the infilling model to generate multiple candidates for the masked position. We then use lower order beam candidates as potential incorrect candidates to generate an incorrect version of the input. We hypothesize that, given the relevant context of a source document, a strong language model generates relevant and factual sequences at higher probabilities, compared to lower probability sequences. For the infilling model, we hypothesize that the lower ranked candidates are often alternative phrases of similar types (in case of entities) or parts-of-speech which are plausible but often not factually correct. Motivated by prior work (Goyal and Durrett, 2020) using lower ranked beam search candidates as a source for adversarial data, we use the lower ranked candidates to construct erroneous summaries from reference summaries.
Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling
Training: Our infilling model M I is trained to take a masked sentence s masked and its relevant context ctx as input and generate a correct phrase to fill in the masked span. To train M I , we construct a dataset using documents d from the training data of existing summarization datasets. For each sentence s in the first-k (k=5) positional sentences of a document d, we identify the subjects, objects and relations {sub, obj, rel} in them using OpenIE (Banko et al., 2007). By iteratively masking each phrase p in {sub,obj,rel}, we create a masked query s masked and its corresponding context ctx by removing the masked sentence from the document, resulting in our training data {s masked , p, ctx}, where p is the masked span text. We train a sequence-to-sequence model M I on this data which takes s masked [SEP] ctx as input and learns to generate p as the output. We intentionally use only sentences from the document as masked queries and do not use sentences from the reference summaries, to ensure that the model does not memorize phrases from the references. Thus, when applied to unseen reference sentences during inference, the model will produces richer beam search candidates.
Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling
We use two news summarization datasets CNN-DailyMail (Hermann et al., 2015) and XSum (Narayan et al., 2018). The two datasets have been extensively studied for the factual consistency in their generated summaries across a variety of models (Goodrich et al., 2019;Cao et al., 2018). Reference summaries from CNN/DM are longer, having on average three sentences, and more extractive in nature. XSum on the other hand has shorter, single sentence summaries and is significantly more abstractive in nature. The summaries in these datasets are qualitatively different, and hence models trained on the two datasets present varied levels of challenges in maintaining factual consistencies.
Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling
Prior work have studied summaries generated from different language models and characterized the factual errors in them (Pagnoni et al., 2021). Generated summaries on the CNN/DM dataset are more extractive in nature and hence are more factual (∼70% of summaries are factual) than the more abstractive generated summaries of XSum (∼20% of summaries are factual). The longer summaries in CNN/DM display discourse level inconsistencies while summaries from XSum often hallucinate content which is not supported by the source document. Hence, the two datasets present a varied setting for evaluating the efficacy of our model across different kinds of errors. For our main evaluation, we evaluate the overall capability of FACTE-DIT in correcting errors in summaries generated by a BART model.
Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling
We further evaluate our model on the FRANK benchmark (Pagnoni et al., 2021) which contains generated summaries obtained using multiple language models for both datasets annotated with human judgements on their factuality and the category of factual error. As different language models have different distribution of factual error types, this evaluation helps us study the generalizability of FACTEDIT in correcting errors across them. 4 For the CNN/DM dataset, it contains model outputs from a LSTM Seq-to-Seq model (S2S) (Rush et al., 2015), a Pointer-Generator Network (PGN) model (See et al., 2017), a Bottom-Up Summarization (BUS) model (Gehrmann et al., 2018), a Bert based Extractive-Abstractive model (BertSum) (Liu and Lapata, 2019) and a jointly pretrained transformer based encoder-decoder model BART . For the XSum dataset, it contains model outputs from a Topic-Aware CNN Model (Narayan et al., 2018), a Pointer-Generator Network (PGN) model, a randomly initialized (TransS2S) (Vaswani et al., 2017) and one initialized with Bert-Base (BertS2S) (Devlin et al., 2019).
Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling
Setup: We use OpenIE (Banko et al., 2007) to pre-process each summary and extract subject, object, predicate triples for each summary sentence. We use BART-base  as our sequence-to-sequence model for the infilling based data generator and the fact correction model. Both models were trained with a batch size of 48, a learning rate of 3e-5, and warm-up of 1000 for 1 epoch. The maximum input sequence length was 512 and maximum output sequence length was 128. Using the infilling data generator, we generate 1233329 negative, 308332 positive examples for CNN/DM and 724304 negative, 181076 positive, examples for XSum as training data for fact correction. Models were trained on 4 Nvidia GeForce GTX TITAN X GPUs and each training run took ∼15 hours. All hyperparameters were chosen based on generated dev set ROUGE-L (Lin, 2004) on each dataset. Evaluation Setup: Evaluating factual consistency of generated summaries is challenging, with relatively recent metrics developed to detect it. These metrics unfortunately do not correlate highly with human judgements yet. We therefore evaluate our model using two metrics -FactCC (Kryscinski et al., 2020) and Ent-DAE (Goyal and Durrett, 2021); each captures different error types. FactCC is a binary classifier, trained on a synthetic, heuristic error dataset, which is better at detecting simple semantic errors like incorrect entities or numbers. Ent-DAE is a classifier trained on synthetic data constructed using the dependency structure of the text. In addition to semantic errors, it is better at detecting more complex discourse-level errors (Pagnoni et al., 2021). We also report ROUGE (Lin, 2004) to evaluate if our model maintains the fluency of summaries. While ROUGE is less correlated with factuality (Pagnoni et al., 2021;Maynez et al., 2020), it helps evaluate if the corrected summary is fluent and aligned with the reference summary. However, with factual corrections of outputs we expect small drops in ROUGE, since generation models were specifically optimized to maximize ROUGE presumably at the expense of factuality.
Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling
Our evaluation has two settings: i) FACTEDIT -correct all generated summaries in the test set and ii) FACTEDIT + FactCC Filter (FF) -using the FactCC metric we identify factually incorrect summaries, and only correct the incorrect ones. Baselines: We compare our approach with (Cao et al., 2020) as the baseline. The baseline uses a heuristic set of rules proposed by Kryscinski et al. (2020) to introduce simple errors (Entity, Number, Date, and Pronoun) in reference summaries and trains a BART-base model for error correction. Comparing our model with (Cao et al., 2020) helps us evaluate the benefit of our Infilling LM based adversarial data generator. 5
Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling
Prior works have shown that improving factual consistency in summaries leads to a drop in ROUGE scores (Maynez et al., 2020;Cao and Wang, 2021;Cao et al., 2020). Our ROUGE results do not drop significantly and are consistent with prior work. These results show that our model does not significantly change the summaries and the corrected summaries contain the relevant information w.r.t. to the source. Table 2 shows results of using FACTEDIT to correct summaries generated by different types of language models using the FRANK benchmark (Pagnoni et al., 2021). We provide the generated summaries collected in the benchmark along with their source document as input to our trained fact corrector. This setting evaluates the generalizability of our adversarial training data in handling different error distributions from different summarization models.
Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling
Our results show our model significantly improves the factuality in generated summaries across 8 out of 9 test models. The FactCC Filter helps improves results in CNN/DM setting but does not change results in XSum similar to results in §4.1. In the more extractive CNN/DM setting, fact correction improves FactCC scores by ∼5.3 points and DAE scores by ∼10.9 points on average across models. In the more challenging and abstractive XSum dataset, we improve FactCC scores by ∼24 points and DAE scores by ∼31 points on average. Our results show that our model trained using Infilling LM based adversarial data is able to generalize and correct errors in generated summaries across different model types. Further, the significant improvement in XSum suggests that using LMs to generate factually incorrect candidates produces rich negative examples which help correct errors in Figure 3: Performance of FactEdit across different error categories in comparison to baseline (Cao et al., 2020) . FACTEDIT improves the percentage of factual summaries across diverse types of factual errors. more abstractive summaries.
Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling
Pretrained models like BART, BertSum and BertS2S have improved generation capabilities and make lesser mistakes in generating the right entity or predicate and more mistakes in discourse structuring (Pagnoni et al., 2021). FACTEDIT correspondingly shows larger improvements in DAE scores than FactCC scores in these pretrained models. The Pointer-Generator model being highly extractive in nature scores highly in factuality metrics in the CNN/DM setting and FACTEDIT reduces results in this setting showing that our model is not beneficial in copy-based model settings. On the other hand, in the XSum setting, the base Pointer-Generator model scores poorly and correcting factual errors in them improves factuality scores. Non-pretrained sequence-to-sequence models like Seq2Seq and TransSeq2Seq score poorly in both ROUGE and Factuality scores due to their limited language generation capabilities. By correcting factual errors in them, we improve factuality metrics significantly without changes in ROUGE, indicating that the gains are due to fact correction and not just rewriting the summary using a strong language model.