Is it possible to leverage LLMs to generate training data which can be used to train smaller, task-specific models?
Recently, with the generative capacity of LLMs,
researchers proposed generating datasets with zero
or very few samples and training a separate model
to serve the specific task (Kumar et al., 2020; Yoo
et al., 2021; Sahu et al., 2022; Yuan et al., 2021;
Hartvigsen et al., 2022). As this approach would
extract information from large models, they would
be analogous to knowledge distillation (Phuong
and Lampert, 2019; Hinton et al., 2015) or dataset
distillation (Wang et al., 2018; Cazenavette et al.,
2022). LLM-generated data has also been used to
test other trained models (Ribeiro and Lundberg,
2022; Perez et al., 2022). In this work, we extend
the previous work by investigating the generation
of high-quality data with accurate diversification
Data Augmentation for Intent Classification with Off-the-shelf Large Language Models
Data augmentation is a widely employed technique to alleviate the problem of data scarcity. In this work, we propose a prompting-based approach to generate labelled training data for intent classification with off-the-shelf language models (LMs) such as GPT-3. An advantage of this method is that no task-specific LM-fine-tuning for data generation is required; hence the method requires no hyper parameter tuning and is applicable even when the available training data is very scarce. We evaluate the proposed method in a few-shot setting on four diverse intent classification tasks. We find that GPT-generated data significantly boosts the performance of intent classifiers when intents in consideration are sufficiently distinct from each other. In tasks with semantically close intents, we observe that the generated data is less helpful. Our analysis shows that this is because GPT often generates utterances that belong to a closely-related intent instead of the desired one. We present preliminary evidence that a prompting-based GPT classifier could be helpful in filtering the generated data to enhance its quality.
[{"first":"Gaurav","last":"Sahu","middle":[]},{"first":"Pau","last":"Rodriguez","middle":[]},{"first":"Issam","last":"Laradji","middle":[]},{"first":"Parmida","last":"Atighehchian","middle":[]},{"first":"David","last":"Vazquez","middle":[]},{"first":"Dzmitry","last":"Bahdanau","middle":[]}]
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
Toxic language detection systems often falsely flag text that contains minority group mentions as toxic, as those groups are often the targets of online hate. Such over-reliance on spurious correlations also causes systems to struggle with detecting implicitly toxic language.To help mitigate these issues, we create ToxiGen, a new large-scale and machine-generated dataset of 274k toxic and benign statements about 13 minority groups. We develop a demonstration-based prompting framework and an adversarial classifier-in-the-loop decoding method to generate subtly toxic and benign text with a massive pretrained language model. Controlling machine generation in this way allows ToxiGen to cover implicitly toxic text at a larger scale, and about more demographic groups, than previous resources of human-written text. We conduct a human evaluation on a challenging subset of ToxiGen and find that annotators struggle to distinguish machine-generated text from human-written language. We also find that 94.5% of toxic examples are labeled as hate speech by human annotators. Using three publicly-available datasets, we show that finetuning a toxicity classifier on our data improves its performance on human-written data substantially. We also demonstrate that ToxiGen can be used to fight machine-generated toxicity as finetuning improves the classifier significantly on our evaluation subset.
[{"first":"Thomas","last":"Hartvigsen","middle":[]},{"first":"Saadia","last":"Gabriel","middle":[]},{"first":"Hamid","last":"Palangi","middle":[]},{"first":"Maarten","last":"Sap","middle":[]},{"first":"Dipankar","last":"Ray","middle":[]},{"first":"Ece","last":"Kamar","middle":[]}]
Data Augmentation using Pre-trained Transformer Models
Language model based pre-trained models such as BERT have provided significant gains across different NLP tasks. In this paper, we study different types of transformer based pre-trained models such as auto-regressive models (GPT-2), auto-encoder models (BERT), and seq2seq models (BART) for conditional data augmentation. We show that prepending the class labels to text sequences provides a simple yet effective way to condition the pre-trained models for data augmentation. Additionally, on three classification benchmarks, pre-trained Seq2Seq model outperforms other data augmentation methods in a low-resource setting. Further, we explore how different pre-trained model based data augmentation differs in-terms of data diversity, and how well such methods preserve the class-label information.
[{"first":"Varun","last":"Kumar","middle":[]},{"first":"Ashutosh","last":"Choudhary","middle":[]},{"first":"Eunah","last":"Cho","middle":[]}]
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
Large-scale language models such as GPT-3 are excellent few-shot learners, allowing them to be controlled via natural text prompts. Recent studies report that prompt-based direct classification eliminates the need for fine-tuning but lacks data and inference scalability. This paper proposes a novel data augmentation technique that leverages large-scale language models to generate realistic text samples from a mixture of real samples. We also propose utilizing soft-labels predicted by the language models, effectively distilling knowledge from the large-scale language models and creating textual perturbations simultaneously. We perform data augmentation experiments on diverse classification tasks and show that our method hugely outperforms existing text augmentation methods. Ablation studies and a qualitative analysis provide more insights into our approach.
[{"first":"Kang","last":"Yoo","middle":["Min"]},{"first":"Dongju","last":"Park","middle":[]},{"first":"Jaewook","last":"Kang","middle":[]},{"first":"Sang-Woo","last":"Lee","middle":[]},{"first":"Woomyeong","last":"Park","middle":[]}]
Data Augmentation for Intent Classification with Off-the-shelf Large Language Models
A key challenge in creating task-oriented conversational agents is gathering and labelling training data. Standard data gathering options include manual authoring and crowd-sourcing. Unfortunately, both of these options are tedious and expensive. Data augmentation is a widely used strategy to alleviate this problem of data acquisition.
Data Augmentation for Intent Classification with Off-the-shelf Large Language Models
There are two particularly promising paradigms for data augmentation in natural language processing that use pretrained language models (LMs) (Peters et al., 2018;Devlin et al., 2018). The first family of methods fine-tunes an LM on task-specific Figure 1: Generation Process. Given a seed intent (here, music_likeness) and K(=10) available examples for that intent, we construct a prompt following the shown template. Note that the last line of the prompt is incomplete (there is no new line character.) We then feed this prompt to a GPT-3 engine, which generates some completions of the prompt. In this example, red text denotes unfaithful examples and blue text denotes faithful examples. Note: For brevity, we only show ten generated sentences. data and generates new examples using the finetuned LM (Wu et al., 2018;Kumar et al., 2019Kumar et al., , 2021Anaby-Tavor et al., 2020;Lee et al., 2021). A limitation of these methods is that, in a realworld scenario, task-specific data is scarce and finetuning an LM can quickly become the bottleneck. The second family of methods sidesteps this bot-tleneck by employing off-the-shelf pretrained LMs such as GPT-3 (Brown et al., 2020) to directly generate text without any task-specific fine-tuning. In particular, data generation by the GPT3Mix approach by Yoo et al. (2021) boosts performance on multiple classification tasks; however, they only consider tasks with few (up to 6) classes and easyto-grasp class boundaries (e.g., positive and negative).
Data Augmentation for Intent Classification with Off-the-shelf Large Language Models
This work studies the applicability of massive off-the-shelf LMs, such as GPT-3 and GPT-J (Wang and Komatsuzaki, 2021) to perform effective data augmentation for intent classification (IC) tasks. In IC, the end goal is to predict a user's intent given an utterance, i.e., what the user of a task-oriented chatbot wants to accomplish. Data augmentation for IC is particularly challenging because the generative model must distinguish between a large number (in practice up to several hundreds) of fine-grained intents that can be semantically very close to each other. Prior methods such as GPT3Mix prompt the model with the names of all classes as well as a few examples from randomly chosen classes. We test GPT3Mix for one and observe that such approaches are poorly suitable for intent classification tasks with tens or hundreds of possible intents. Instead, in this study, we use a simple prompt structure that focuses on a single seed intent (see Figure  1) as it combines the intent's name and all available examples.
Data Augmentation for Intent Classification with Off-the-shelf Large Language Models
Our experiments primarily focus on few-shot IC on four prominent datasets: CLINC150 (Larson et al., 2019), HWU64 (Xingkun Liu and Rieser, 2019), Banking77 (Casanueva et al., 2020), and SNIPS (Coucke et al., 2018). We also consider a partial few-shot setup to compare to the Example Extrapolation (Ex2) approach by Lee et al. (2021) who use a similar prompt but fine-tune the LM instead of using it as is. The main findings of our experiments are as follows: (1) GPT-generated samples boost classification accuracy when the considered intents are well-distinguished from each other (like in CLINC150, SNIPS).
Data Augmentation for Intent Classification with Off-the-shelf Large Language Models
(2) On more granular datasets (namely HWU64 and Banking77), we find that GPT struggles in distinguishing between different confounding intents. (3) A smallscale study to further understand this behaviour suggests that GPT could be used as a classifier to filter out unfaithful examples and enhance the quality of the generated training set. Additionally, we investigate how valuable the generated data could be if relabelled by a human. Using an oracle model, we show that (4) the human labelling of GPT-generated examples can further improve the performance of intent classifiers, and that (5) LMgenerated data has a higher relabelling potential compared to edit-based augmentation techniques, such as Easy Data Augmentation (EDA) (Wei and Zou, 2019).
Data Augmentation for Intent Classification with Off-the-shelf Large Language Models
We consider training an intent classifier, where an intent is a type of request that the conversational agent supports; e.g. the user may want to change the language of the conversation, play a song, transfer money between accounts, etc. However, collecting many example utterances that express the same intent is difficult and expensive. Therefore, this paper experiments with a straightforward method to augment the training data available for an intent: creating prompts from the available examples and feeding them to a large language model such as GPT-3 (Brown et al., 2020). Figure 1 illustrates the process of data generation for an intent with K available examples.
Data Augmentation for Intent Classification with Off-the-shelf Large Language Models
We use four intent classification datasets in our experiments with varying levels of granularity among intents. CLINC150 (Larson et al., 2019), HWU64 (Xingkun Liu and Rieser, 2019) are multidomain datasets, each covering a wide range of typical task-oriented chatbot domains, such as playing music and setting up alarms. Importantly, the CLINC150 task also contains examples of out-of-scope (OOS) utterances that do not correspond to any of CLINC's 150 intents. Bank-ing77 (Casanueva et al., 2020) is a single domain dataset with very fine-grained banking-related intents. Finally, the SNIPS (Coucke et al., 2018) dataset contains 7 intents typical for the smart speaker usecase. We refer the reader to Table 1 for exact statistics of all used datasets.
Data Augmentation for Intent Classification with Off-the-shelf Large Language Models
The main data-scarce setup that we consider in this work is the few-shot setup, where only K = 10 training examples are available for every intent of interest. Additionally, to compare to example extrapolation with fine-tuned language models as proposed by Lee et al. (2021), we consider a partial  few-shot setup. In the latter setting, we limit the amount of training data only for a handful of fewshot intents 2 and use the full training data for others. When data augmentation is performed, we augment the few-shot intents to have N examples, where N is the median number of examples per intent of the original data.
Data Augmentation for Intent Classification with Off-the-shelf Large Language Models
To precisely describe the training and test data in all settings, we will use D part to refer to dataset parts, i.e. train, validation, and test. In addition, we use D F and D M to refer to data-scarce and data-rich intents (the latter only occur in the partial few-shot setting). This notation is defined for all parts, therefore, D part = D {F,part} ∪ D {M,part} , ∀ part ∈ {train, val, test}. When GPT-3 or a baseline method is used to augment the training data we generate N − K examples per intent and refer to the resulting data asD F,train . We experiment with four different-sized GPT-3 models 3 by OpenAI and GPT-J by EleutherAI 4 to obtainD. The four GPT-3 models are: Ada, Babbage, Curie, and Davinci. In order, Ada is the smallest model and Davinci is the largest. Model sizes of GPT-3 engines are not known precisely but are estimated by Eleuther AI to be between 300M and 175B parameters 5 . 2 We use the truncation heuristic provided by Lee et al. (2021): https://github.com/google/example_ extrapolation/blob/master/preprocess_ clinc150.py 3 https://beta.openai.com/docs/engines 4 https://github.com/kingoflolz/ mesh-transformer-jax/ 5 https://blog.eleuther.ai/ gpt3-model-sizes/
Data Augmentation for Intent Classification with Off-the-shelf Large Language Models
We fine-tune BERT-large (Devlin et al., 2018) on the task of intent classification by adding a linear layer on top of the [CLS] token (Wolf et al., 2019). In all setups we use the original validation set for tuning the classifier's training hyperparameters. We chose to use the full validation set as opposed to a few-shot one to avoid issues with unstable hyperparameter tuning and focus on assessing the quality of the generated data.
Data Augmentation for Intent Classification with Off-the-shelf Large Language Models
Full few-shot. In this setup, we treat all the intents as few-shot and evaluate our method on the following three scenarios: (i) Baseline: all the intents are truncated to K = 10 samples per intent, (ii) Augmented:D {F,train} is generated using GPT and models are trained on D {F,train} ∪ D {F,train} and (iii) EDA-baseline: same as above, butD {F,train} is generated using Easy Data Augmentation (EDA) by Wei and Zou (2019). For each scenario, we report the 1) overall in-scope accuracy on the complete test set D test , i.e. intent classification accuracy excluding OOS samples in the test set, and 2) few-shot classification accuracy of the models on D {F,test} . For CLINC150, we also report out-of-scope recall (OOS recall) on D test that we compute as the percentage of OOS examples that the model correctly labelled as such.
Data Augmentation for Intent Classification with Off-the-shelf Large Language Models
The purpose of this setting is to estimate what further gains can be achieved if the data generated by GPT were labelled by a human. We train an oracle O on the full training data D train . We also use O to assess the quality of the generated data. Namely, we compute fidelity of the generated data as the ratio of generated utterances that the oracle labels as indeed belonging to the intended seed intent. A higher fidelity value means that the generated samples are more faithful to original data distribution.
Data Augmentation for Intent Classification with Off-the-shelf Large Language Models
Partial few-shot. In this setup, we train S intent classifiers, choosing different few-shot intents every time to obtain D F . We then average the metrics across these S runs. For CLINC150, S = 10 corresponding to the 10 different domains, whereas for SNIPS, S = 7 corresponding to the 7 different intents. We evaluate our method on the following three scenarios introduced by Lee  For each scenario in this setup, we report the overall in-scope classification accuracy (and OOS Recall for CLINC150).
Data Augmentation for Intent Classification with Off-the-shelf Large Language Models
For both partial few-shot and full few-shot settings, we report means and standard deviations over 10 repetitions of each experiment.
Data Augmentation for Intent Classification with Off-the-shelf Large Language Models
Full few-shot. Temp. CLINC150 and SNIPS. Importantly, using larger GPT models for data augmentation brings significantly bigger gains. Data augmentation results on Banking77 and HWU64 are, however, much worse, with no or little improvement upon the plain fewshot baseline. We present a thorough investigation of this behaviour in Section 4.1. One can also see that data augmentation with GPT models lowers the OOS recall. Next, we observe that relabelling EDA and GPTgenerated sentences by the oracle gives a significant boost to accuracies across the board, confirming our hypothesis that human inspection of generated data could be fruitful. Importantly, we note that the magnitude of improvement for EDA is less than for GPT models. This suggests that GPT models generate more diverse data that can eventually be more useful after careful human inspection. Lastly, relabelling also improves OOS recall on CLINC150, which is due to the fact that much of the generated data was labelled as OOS by the oracle.
Data Augmentation for Intent Classification with Off-the-shelf Large Language Models
Partial few-shot.    (Lee et al., 2021), both in terms of absolute accuracies and the relative gains brought by data augmentation. Note that Ex2 uses T5-XL (Roberts et al., 2020) with nearly 3 billion parameters as its base intent classifier, while our method uses BERT-large with only 340 million parameters.
Data Augmentation for Intent Classification with Off-the-shelf Large Language Models
Effect of GPT sampling temperature. We investigate the impact of generation temperature on the quality and fidelity of generated data. We perform this investigation on the CLINC150 dataset using the partial few-shot setup. Results in Figure 2 show that, for all engines, the generated data leads to the  highest classification accuracy when the generation temperature is around 1.0, although lower temperatures result in higher OOS recall. We also observe that the fidelity of the generated samples decreases as we increase the temperature (i.e. higher diversity, see Figure 2c). This suggests that higher fidelity does not always imply better quality samples as the language model may simply copy or produce less diverse utterances at lower temperatures. In Appendix A, we perform a human evaluation, reaching similar conclusions as when using an oracle to approximate fidelity.
Data Augmentation for Intent Classification with Off-the-shelf Large Language Models
Fidelity on different datasets. Our results in Section 4 show that data augmentation gains are much higher on CLINC150 and SNIPS than on HWU64 and Banking77. To contextualize these results, we report the fidelity of GPT-J-generated data for all these tasks in Figure 3b. Across all generation temperatures, the fidelity of the generated data is higher for CLINC150 and SNIPS than for HWU64 and Banking77. For all datasets, the fidelity is higher when the generation temperature is lower; however, Figure 3a shows that low-temperature data also does improve the model's performance.
Data Augmentation for Intent Classification with Off-the-shelf Large Language Models
Data generation for close intents. To better understand the lower fidelity and accuracy on HWU64 and Banking77 datasets, we focus on intents with the lowest fidelities. Here, by intent fidelity, we mean the percentage of the intent's generated data that the oracle classified as indeed belonging to the seed intent. In the Banking77 dataset, the lowest-fidelity intent is "topping_up_by_card." For this intent, only 33% of the Davinci-generated sentences were labelled as "topping_up_by_card"   Figure 4: Distribution of labels as predicted by the oracle for lowest-fidelity intents in Banking77 and HWU64 datasets ("topping_up_by_card" and "mu-sic_likeness," respectively). Green areas denote the portion of generated sentences deemed fit by the oracle for the lowest-fidelity intents in the two datasets. Red and Blue areas respectively correspond to the most common and the second most common alternative intent predicted by the oracle.
Data Augmentation for Intent Classification with Off-the-shelf Large Language Models
by the oracle, implying that two-thirds of the sentences did not fit that intent, "top_up_failed" and "top_up_card_charge" being the two most common alternatives chosen by the oracle. Similarly, only 50% of the Davinci-generated sentences abide by the lowest-fidelity "music_likeness" intent in the HWU64 dataset, "play_music" and "general_quirky" being the most common intents among the "unfaithful" sentences. Figure 4 visualizes this high percentage of unfaithful generated sentences. It also shows the proportion of the two most common alternatives that the oracle preferred over the seed intent. Table 4 presents generated sentences for confounding intents in the HWU64 and Banking77 datasets. There are clear indications of mix-up of intents, e.g., Davinci generates, "play a song with the word honey," which should belong to "play_music" rather than "music_likeness." There are also instances where the LM mixes two intents; for instance, Davinci generates "Hi my app is activated on activate.co.in, but unable to top up my phone. I tried credit card, debit card and Paytm but fails," which could belong to either "topping_up_by_card" intent (as it mentions about using credit card in the context of a top up) or "top_up_failed" (as the top up ultimately fails).
Data Augmentation for Intent Classification with Off-the-shelf Large Language Models
We perform extra investigations to better understand what limits GPT-3's ability to generate data accurately. We hypothesize that one limiting factor can be GPT-3's inability to understand fine-grained differences in the meanings of utterances. To verify this hypothesis, we evaluate how accurate GPT-3 is at classifying given utterances as opposed to generating new ones. Due to the limited prompt size of 2048 tokens, we can not prompt GPT-3 to predict all the intents in the considered datasets. We thus focus on the close intent triplets from HWU64 and Banking77 datasets that we use in Table 4. We compare the 3-way accuracy of a prompted GPT-3 classifier to the similarly-measured 3-way performance of conventional BERT-large classifiers. We prompt GPT-3 with 10 examples per intent (see Figure 5). For comparison, we train BERT-large classifiers on either the same 10 examples or the full training set. Table 5 shows that the Davinci version of GPT-3 performs in between the 10-shot and the full-data conventional classifiers. This suggests that while GPT-3's understanding of nuanced intent differences is imperfect, it could still be sufficient to improve the performance of the downstream few-shot model. Inspired by this finding, we experiment with using GPT-3's classification abilities to improve the quality of generated data. Namely, we reject the generated utterances that GPT-3 classifies as not belonging to the seed intent. For both HWU64 and Banking77, this filtering method significantly improves the fidelity of the generated data for the chosen close intent triplets.
Data Augmentation for Intent Classification with Off-the-shelf Large Language Models
To test our initial hypothesis that prior methods such as GPT3Mix are not suitable for intent classification, we experiment with the said method on the CLINC150 dataset using Curie. Specifically, we include an enumeration of the 150 intent names in the prompt and randomly select one example for K intents. We observe a poor in-scope accuracy of 86.33% in the Augmented scenario 6 . Furthermore, the generated samples have low fidelity (27.96%).
Data Augmentation for Intent Classification with Off-the-shelf Large Language Models
We also test a mixture of GPT3Mix prompt and our prompt where we include all the K examples for the seed intent instead of 1 example per K randomly sampled intents. This mixed variant also performs poorly on CLINC150 and only achieves an in-scope accuracy of 86.05% 7 and a fidelity of 33.56%. Our interpretation of this result is that GPT cannot handle the long list of 150 intent names in the prompt.
Data Augmentation for Intent Classification with Off-the-shelf Large Language Models
Each example in the following list contains a sentence that belongs to a category. A category is one of the following: music_likeness, play_music, music_settings: sentence: next i want to hear shinedown ; category: play_music sentence: i am the living blues ; category: music_likeness sentence: open music player settings ; category: music_settings sentence: play hopsin from my latest playlist ; category: play_music sentence: i like this song ; category:
Data Augmentation for Intent Classification with Off-the-shelf Large Language Models
play_music,music_likeness,music_settings, music_likeness,music_likeness,help_command Figure 5: Using GPT-3 as a classifier. Given a triplet of close intents, we mix and shuffle the multiple seed examples available for each of them. Then, we append an incomplete line to the prompt with just the generated sentence and feed it to GPT-3 multiple times. Among the responses, we choose the most generated in-triplet intent as the predicted intent ("music_likeness" in the above example). Note: For brevity, we don't show all the seed examples and predictions. feature hundreds of intents (see Section 4.3). Therefore, we choose a different prompt that encourages the model to extrapolate between examples of a seed intent similarly to (Lee et al., 2021).
Data Augmentation for Intent Classification with Off-the-shelf Large Language Models
Other work on few-shot intent classification explores fine-tuning dialogue-specific LMs as classifiers as well as using similarity-based classifiers instead of MLP-based ones on top of BERT (Vulić et al., 2021). We believe that improvements brought by data augmentation would be complementary to the gains brought by these methods.
Data Augmentation for Intent Classification with Off-the-shelf Large Language Models
Lastly, our method to filter out unfaithful GPT generations is related to the recent work by Wang et al. (2021) that proposes using GPT3 for data labelling. A crucial difference with respect to our work, however, is that we use GPT-3 for rejecting mislabelled samples rather than proposing labels for unlabelled samples.
Data Augmentation for Intent Classification with Off-the-shelf Large Language Models
We propose a prompt-based method to generate intent classification data with large pretrained lan-guage models. Our experiments show that generated data can be helpful as additional labelled data for some tasks, whereas, for other tasks, the generated data needs to be either relabelled or filtered to be helpful. We show that a filtering method that recasts the same GPT model as a classifier can be effective. Our filtering method, however, requires knowing the other intents that the generated data is likely to belong to instead of the seed intent. Future work can experiment with heuristics for approximately identifying the most likely actual intents for the generated utterances. This would complete a data generation and filtering pipeline that, according to our preliminary results in Section 4.2 here, could be effective. Other filtering methods could also be applied, such as looking at the likelihood of the generated utterances as explored in a concurrent work by Meng et al. (2022). Lastly, an interesting future work direction is identifying which generated utterances most likely need a human inspection.
Data Augmentation for Intent Classification with Off-the-shelf Large Language Models
As discussed for the GPT3Mix method in Yoo et al. (2021), using large language models for data augmentation presents several challenges: they exhibit social biases and are prone to generating toxic content. Therefore, samples generated using our prompting-based approach need to be considered carefully.
Data Augmentation for Intent Classification with Off-the-shelf Large Language Models
To address such ethical concerns, human inspection would be the most reliable way to identify and filter out problematic generations. The practitoners who apply our method may also consider debiasing the language model before using it for generation (Schick and Schütze, 2021).
Data Augmentation for Intent Classification with Off-the-shelf Large Language Models
In Figure 2 we evaluate the fidelity of the samples generated by GPT-3 with respect to the original set of sentences used to prompt it. Fidelity is approximated by the classification performance of an "oracle" intent classifier trained on the whole dataset (D train ∪ D test ) and evaluated over the generated samples. In order assess whether the oracle predictions are comparable to those of a human, we perform a human evaluation study.  Figure 6: Human evaluation. Error rate of human evaluators at the task of finding whether any sentence in a group of 5 was generated by GPT-3 or not. Each color represents a different GPT-3 engine. Higher error rate indicates that humans could not correctly identify generated samples and thus it also indicates higher fidelity. The standard error is displayed as a vertical line on top of each bar. We consider that a model produces sentences with high fidelity if a human is unable to distinguish them from a set of human-generated sentences be-longing to the same intent. Therefore, for each intent in the CLINC150 dataset, we sample five random examples and we randomly choose whether to replace one of them by a GPT-3 generated sentence from the same intent. We generate sentences with each of the four GPT-3 models considered in the main text with two different temperatures (0.8 and 1.0). The sentence to replace is randomly selected. Finally, the five sentences are displayed to a human who has to choose which of the sentences is generated by GPT-3, if any.
Data Augmentation for Intent Classification with Off-the-shelf Large Language Models
The task is presented to human evaluators in the form of a web application (see Figure 7). We placed a button next to each sentence in order to force human evaluators to individually consider each of the examples. Once annotated, the evaluator can either submit, discard, or leave the task to label later. We used a set of 15 voluntary evaluators from multiple backgrounds, nationalities, and genders. Each evaluator annotated an average of 35 examples, reaching a total of 500 evaluated tasks.
Data Augmentation for Intent Classification with Off-the-shelf Large Language Models
For each model and temperature, we report the error rate of humans evaluating whether a task contains a GPT-generated sample. We consider that evaluators succeeds at a given task when they correctly find the sentence that was generated by GPT or when they identify that none of them was generated. Thus, the error rate for a given model and temperature is calculated as #failed / total_evaluated.
Data Augmentation for Intent Classification with Off-the-shelf Large Language Models
Results are displayed in Figure 6. We find that human evaluators tend to make more mistakes when the temperature used to sample sentences from GPT-3 is smaller. This result is expected since lowering the temperature results in sentences closer to those prompted to GPT-3, which are humanmade. We also observe that models with higher capacity such as Davinci tend to generate more indistinguishable sentences than lower-capacity models such as Ada, even for higher temperatures. These results are in agreement with the "oracle" fidelity results introduced in Figure 2.
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
Toxic language detectors often over-rely on minority identity mentions 1 when flagging a statement as toxic, without considering the deeper semantic meaning of the statement (Dixon et al., 2018;Röttger et al., 2021). This can lead to severe underdetection of subtle hate (e.g., "They have been bred to be good at sports and entertainment, but not much else"; Figure 1) and over-detection of benign statements (e.g., "child abuse is wrong, racism is wrong, sexism is wrong"; Figure 1). Importantly, such biases in toxicity detection risk further marginalizing or censoring minority groups (Yasin, 2018;Sap et al., 2019;Dias Oliva et al., 2020;Are, 2020;Díaz and Hecht-Felella, 2021).
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
We introduce TOXIGEN, a large-scale machinegenerated dataset of 274,186 toxic and benign statements. To create this dataset, we leverage the massive pretrained language model GPT-3 (Brown et al., 2020), which is known to produce closeto-human-like text (Clark et al., 2021;Dou et al., 2021) but also easily generates socially biased and toxic content (Sheng et al., 2019;Gehman et al., 2020). While such human-like bias and toxicity poses real threats, we use this undesirable behavior in models like GPT-3 to improve existing toxic language classifiers, providing a path forward for mitigating systemic bias. Created using demonstrationbased prompting and pretrained toxicity classifiers, TOXIGEN covers over 135k toxic and 135k benign statements about 13 minority identity groups (e.g., African Americans, women, LGBTQ+ folks, etc.).
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
Using this machine generated approach has two advantages over scraping posts from the web as done by previous work (e.g., Davidson et al., 2017;Founta et al., 2018;Zampieri et al., 2019). First, it allows us to limit spurious identity-toxicity correlations (Dixon et al., 2018;Zhou et al., 2021) by generating equal numbers of toxic/benign statements for each demographic group, including those that are often overlooked in toxic language corpora (e.g., Native Americans). Second, machine generation and careful prompting enables us to generate implicit toxicity (i.e., without swearwords or slurs), which is by definition hard to detect or find and thus often missing in toxic language corpora (Wiegand et al., 2021). Indeed, 98.2% of TOXIGEN statements are implicit, i.e., devoid of explicit profanity, generator classifiers ALICE
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
You can tell if a woman is a good person or not by how much she covers her skin
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
If you have ever been to a restaurant, you have probably noticed that the service is much better if the waiter is white, and the food is much better if the chef is white
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
They have been bred to be good at sports and entertainment, but not much else.
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
Targeted adversarial generation with ALICE the only thing that can be done to improve the situation is to put them all in chains and make them work in the fields for the rest of their lives
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
If you are a woman, you should cover your hair, because it is a sign of your submission to your husband, who is your master Yes, you should child abuse is wrong, racism is wrong, sexism is wrong
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
There is a big difference between a muslim and a terrorist 73% toxic it is wrong to think that a person with a dark skin tone will be a violent and uneducated person 94% unsafe if you have a beard, I can not conclude that you are a terrorist 97% toxic in the past the only way to get a job for a black person was to be a slave which was not fair for the black people
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
It's wrong Toxic Benign Figure 1: Examples of statements that fool Google's Perspective API ( ), HateBERT ( ), Open AI content filter ( ), AI2 Delphi ( ), 4 and Roberta ( ). Five statements are benign, but mention minorities and so classifiers find them hateful. Five are toxic sentences, but the classifiers find them neutral. ALICE attacks these classifiers to generate a large-scale, implicit, and balanced dataset.
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
slurs, or swearwords (Table 1).
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
To generate a challenging subset of TOXIGEN, we introduce ALICE, 2 an adversarial classifier-inthe-loop decoding algorithm. We use ALICE to control the toxicity of output text by pitting a toxicity classifier against a text generator during beam search decoding. Given a toxic prompt, we can encourage generations to be less toxic based on the classifier scores. Similarly, we can steer a language model with neutral prompting towards higher toxicity generations. Our experiments with five publicly-available toxicity classifiers show that the generated sentences in both cases above fool toxicity classifiers (see Figure 1).
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
We validate the quality of our machine-generated dataset through a comprehensive human evaluation. Our results show that on a sample of 792 machinegenerated sentences, 90% could be mistaken for human-written text. We also find that the generated data indeed contains a wide variety of specific references to the minority groups mentioned in the prompts ( §4.2). This indicates that our data generation approaches (with or without ALICE) successfully control the generation towards the desired toxicity and minority group mention.
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
Further experimental results demonstrate that 2 Adversarial Language Imitation with Constrained Exemplars 4 Delphi does not produce toxicity probabilities, so we use Open AI's content filter to game Delphi. A Delphi author has confirmed probabilities will be available soon.
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
fine-tuning existing classifiers on TOXIGEN consistently improves performance (+7-19%) on 3 existing human-written implicit toxic datasets: Im-plicitHateCorpus (ElSherief et al., 2021), SocialBi-asFrames (Sap et al., 2020), andDynaHate (Vidgen et al., 2021). This indicates that the dataset generated in this work and the approaches for generating data provide major steps towards improving toxicity classifiers, and could potentially be used downstream to address the issues from biased machine generation (Sheng et al., 2019) or neutral toxic degeneration (Gehman et al., 2020).
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
We release our code and the TOXIGEN dataset publicly. 3 We also include two models pretrained on TOXIGEN along with our human evaluations.
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
Detecting implicit toxicity about minority groups (e.g., stereotyping, microaggressions), remains an elusive goal for NLP systems (Han and Tsvetkov, 2020;Wiegand et al., 2021). One key challenge is that, in contrast to explicit toxicity, implicit toxicity is not marked by the use of profanity or swearwords, is sometimes positive in sentiment, and is generally harder to detect or collect at scale (MacAvaney et al., 2019;Breitfeller et al., 2019). Nonetheless, implicitly toxic language about minority or marginalized groups is often psychologically damaging to members of those groups (Sue et al., 2007 Nadal et al., 2014;Kanter et al., 2017;Nadal, 2018;Saleem and Anderson, 2013) and can reinforce stereotypical or hateful perceptions of them (Behm-Morawitz and Mastro, 2008;Soral et al., 2018). A second challenge for detecting subtle toxicity about minority groups is that minority mentions are more often the targets of social biases and toxicity (Hudson, 2017). As such, minority mentions often co-occur with toxicity labels in datasets scraped from online platforms (Dixon et al., 2018). For example, over 93% of mentions of Jewish folk in Sap et al. (2020) are toxic (Wiegand et al., 2021). In turn, models trained on such data can exploit these spurious minority-toxicity correlations instead of considering the deeper semantics of text (Zhou et al., 2021). Importantly, the spurious correlations are also learned by large language models, which are known to produce stereotypical, biased, or toxic content when prompted with minority mentions (Sheng et al., 2019). Given that the main mitigation approach to prevent Large Language Models (LLM) from generating toxic language is to train new classifiers to detect such language, these classifiers also learn the spurious correlations and start blocking most language referencing minority groups. This risks erasure (Xu et al., 2021).
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
With TOXIGEN, we aim for generating a large scale dataset that represent implicit toxicity while balancing between toxic and benign statements, to address the gaps of previous work. As shown in Table 1, existing datasets contain large amounts of explicit toxicity. While valuable, most previous work has relied on scraping data from online platforms, which leads to dataset imbalances with respect to minority-mentioning posts that are toxic vs. benign. Examples are collected at scale using keyword-based scraping approaches (Waseem, 2016;Davidson et al., 2017;Zampieri et al., 2019), the bootstrapped scraping approaches (Founta et al., 2018), and machine-vs-human adversarial data collection (Dinan et al., 2019;Vidgen et al., 2021), among others. In contrast, using large language models to generate our dataset allows us to control the minority groups mentioned in our statements, as well as their implicitness, at larger scale.
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
To create TOXIGEN, we use demonstration-based prompting for LLMs, encouraging a text generator to produce both toxic and benign sentences that mention minority groups without using explicit language. We introduce a classifier-in-the-loop decoding method based on constrained beam search, ALICE, which, along with samples generated without ALICE, contributes to generating a challenging subset of TOXIGEN. Using these methods, we generate a massive set of statements (over 274,000) containing equal numbers of toxic and benign sentences for 13 identity groups-see Table 2.
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
TOXIGEN is generated by prompting a language model to produce both benign and toxic sentences that (1) include mentions of minority groups by name and (2) contain mainly implicit language, which does not include profanity or slurs. To achieve this, we perform demonstration-based prompt engineering: Acquiring example sentences,  passing them to a LLM, 4 and collecting the subsequent response. For example, when shown ten sentences mentioning Muslim folks neutrally, GPT-3 often responds with a similar neutral sentence.
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
Prompts are text fragments passed into language models that can encourage certain behaviors (Brown et al., 2020). However, designing prompts is notoriously challenging (Liu et al., 2021c). While there are several approaches for prompting pretrained LLMs (Liu et al., 2021b), a recent and promising direction is demonstration-based prompting (Gao et al., 2021;Mishra et al., 2021).
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
Here, example statements are passed to an LLMs, encouraging it to produce a similar, but distinct, statement. For example, to encourage LLMs to produce benign statements about Asian people, we might use the following examples: (1) many impactful people are of asian descent, (2) there are hundreds of asian cultures. To engineer prompts that lead to high-quality, group-mentioning toxic and benign statements at scale, we first gather and curate sets of examples.
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
To generate both benign and toxic responses from LLMs that mention minority groups, we first col-lect many examples. Intuitively, given many examples of benign sentences that mention one particular group, a language model can be used to produce more. For benign prompts, we encourage realistic text generation and include diverse voices by collecting benign sentences from blog posts and news articles that mention a group. However, finding large amounts of such data at scale is challengingthis is why implicit datasets are hard to acquire. To build a large enough set of demonstrations, we begin with a small number of examples from the wild, then engage a human-in-the-loop process: collect some demonstrations, pass them to our LLM, comb through many responses, and add the best examples to a growing set. Ensuring that a set of examples consistently produces benign responses that still mention the targeted minority group is challenging and so we iterate this loop many times, sampling random subsets of our examples to serve as prompts and observing the responses. This way, we collect 20-50 demonstration sentences per group, all of which we release.
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
To encourage implicit toxicity from a LLM, we find examples of human-written sentences with implicit toxicity towards each group from hate forums (de Gibert et al., 2018) and Reddit (Breitfeller et al., 2019). We repeat the human-in-the-loop process to expand our sets of examples. Overall, by repeating this process for both toxic and benign examples for all 13 target groups, we create 26 sets of prompts,  with two (benign and toxic) per target group.
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
Demonstration-based prompting alone consistently produces toxic and benign statements about minority groups (see Section 4). There is no guarantee that these statements will be challenging to existing toxicity detectors. Therefore, we also develop ALICE, a variant of constrained beam search (CBS; Anderson et al., 2017;Hokamp and Liu, 2017;Holtzman et al., 2018;Lu et al., 2021) during decoding that generates statements that are adversarial to a given pre-trained toxicity classifier. ALICE creates an adversarial game between a pre-trained language model (PLM) and a toxicity classifier (CLF) during constrained beam search decoding. In many CBS settings, constraints are added during beam search decoding to force the model to either include or exclude a specific word or group of words in the output (Anderson et al., 2017;Hokamp and Liu, 2017;Lu et al., 2021). With ALICE, we instead want to enforce soft constraints on the probabilities coming from a given toxicity classifier CLF during beam search: 5
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
Here, λ L and λ C denote hyperparameters that determine the respective contribution of the language model and classifier to the decoding scoring function. By using this weighted combination, we can steer generations towards a higher or lower probability of toxicity without sacrificing coherence enforced by the language model. To create examples that challenge existing toxicity classifiers, we use two adversarial setups:
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
• False negatives: We use toxic prompts to encourage the language model to generate toxic outputs, then maximize the classifier's probability of the benign class during beam search.
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
• False positives: We use benign prompts to encourage the language model to generate nontoxic outputs, then maximize the probability of the toxic class during beam search.
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
In the first approach, we are also able to detoxify model outputs when the classifier successfully steers the generations towards non-toxic language. ALICE is illustrated in Figure 2.
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
We generate TOXIGEN data with and without ALICE. Without ALICE, we use top-k decoding (Fan et al., 2018) alone with our toxic and benign prompts. With ALICE, we use the HateBERT finetuned OffensEval model from Caselli et al. (2021) as the toxicity classifier (CLF). This model covers a range of direct and veiled offense types. We use GPT-3 for the language model. For decoding, we use λ L = λ C = 0.5, a maximum generation length of 30 tokens, a beam size of 10, and a temperature of 0.9. Due to limitations imposed by the OpenAI GPT-3 API on accessing log probabilities for the full model vocabulary, we restricted the vocabulary   Figure 3: Comparing the proportion of identity group mentions that were desired based on the prompts vs. that were generated, in our annotated evaluation set.
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
We include the actual proportions as data labels.
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
size to the top 100 tokens, and then resample from the "allowed" tokens (tokens not appearing in the prompt) using top-k. 6
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
Statistics of TOXIGEN are presented in Table 2.
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
In our final dataset, generation length varies significantly and, as expected, almost all the statements are implicit. As we show in §4, the ALICEgenerated data is successful at attacking the given toxicity classifier, contributing a challenging, adversarial subset of TOXIGEN. 7 In the released data, we split off a test set that is validated by human annotators (see §4.2).
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
To ensure the quality of TOXIGEN, we conduct human validation experiments and create TOXIGEN-HUMANVAL, a human-validated test set. Specifically, we investigate the reliability of our promptbased and ALICE-based methods at generating human-like statements and controlling statements' toxicity and the minority groups mentioned ( §4.2). Additionally, we measure the effectiveness of AL-ICE-generated statements (vs. top-k-generated) at fooling classifiers ( §4.3).
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
For each generated statement, we ask the annotators various questions, described below, that take into account multiple dimensions of how toxic 6 We force beam search decoding to not use tokens from the prompt to prevent direct copying. Certain tokens appearing in the prompt such as punctuation are allowed. 7 We compute the % of implicit samples using https: //github.com/RobertJGabriel/Googleprofanity-words, the same as ElSherief et al. (2021), also removing ambiguous terms (e.g., "bloody"). machine-generated language presents a potential harm to readers. See Appendix B for an annotation screenshot and other study details.
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
Perceived hatefulness with respect to human or AI-authored text. We first ask annotators to guess whether the statement's author was a human or an AI system (HUMANORAI). Then, we ask whether the statement would be harmful to anyone if an AI system wrote it (HARMFULIFAI), as well as if a human wrote it (HARMFULIFHUMAN); we hypothesize that readers may have different standards for machine-generated text than humanwritten text. For all questions measuring harmfulness of text, we consider potential harm on a 1-5 scale with 1 being clearly benign and 5 indicating very offensive or abusive text.
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
Perceived intent of the writer. We ask readers whether statements were likely intended to be harmful (HARMFULINTENT), since some biased statements can be positively intended (e.g., benevolent sexism; Glick and Fiske, 1996). Additionally, we ask if the statement exhibits a positive stereotype (POSSTEREO), which is also harmful (e.g., model minority myths; Cheryan and Bodenhausen, 2000).
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
To better understand how harm may be perpetrated against the minority group, we ask readers in-depth questions about text's content, following Sap et al. (2020) and Olteanu et al. (2018). We ask whether or not the statement is lewd or sexual (LEWD), whether and how it references the targeted group or other groups (WHICHGROUP, GROUPFRAMING), whether it claims to be factual or opinion (FACTOROPINION).
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
Data and Setup. We selected 792 statements from TOXIGEN to include in our test set, such that no training statement had cosine similarity above 0.7 with any test statement. Each test statement was then rated by 3 annotators from a pool of 156 prequalified annotators from Amazon MTurk (See Appendix B for details).
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
Inter-annotator agreement. To investigate the quality of our annotations, we compute agreement on toxicity ratings. 8 We find that annotators agreed moderately and are higher than or equal rates to prior work on hate speech annotation (Ross et al.,  Table 3: Example responses from human evaluation where machine-generated text fools annotators into thinking the writer is human. Average toxicity scores are on a 1-5 scale (1 being benign and 5 being clearly offensive), and are averaged across annotator responses. We report scores for the case where annotators assume the writer/speaker is AI and the writer/speaker is human respectively.  Fleiss, 1971) and Krippendorff's α=0.64 (Krippendorff, 1980). In 55.17% of cases, all 3 annotators agree, while a majority (≥2/3) agree for 93.4%.
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
Human validation results. First, we find that our machine-generated statements are largely indistinguishable from human-written statements. For example-see Table 3-human annotators often predict that our text is generated by a human. In fact, on average 90.5% of machine-generated examples are thought to be human-written by a majority of annotators, as shown in Figure 4. We also note that harmful text confuses readers slightly more than non-harmful text: 92.9% of toxic examples are mislabeled as human-written compared to 90.2% for non-toxic. Most toxic examples are also hate speech (94.56%). While opinions are common in both toxic and non-toxic examples, most fact-claiming text is non-toxic.
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
Second, we find that demonstration-based prompting reliably generates toxic and benign statements about minority groups ( §4.3). Further, for the machine-generated examples, we find that 30.2% are harmful (given a score of >3), while only 4% are ambiguous. This indicates that these data are sufficiently toxic or benign. We also find that all identity groups covered by the dataset were represented in the human study (see Figure 3), and observe that the identity group referenced by the prompt is generally the same as the group referenced by the corresponding TOXIGEN text, though there is some deviation. This is likely due to GPT-3 conflating identities or mentioning multiple groups.
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
Interestingly, there is no significant difference in toxicity when we account for whether annotators perceive scores as written by humans or AI ( Figure 5). This finding indicates that our machinegenerated text is perceived as similarly harmful to human text. We also find that the most common framing tactic is "moral judgement", or questioning the morality of an identity group, which has been linked to toxicity by prior work (Hoover et al., 2019).
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
As further validation, we investigate whether AL-ICE-generated statements are more adversarial compared to top-k-generated ones. For 125 randomlyselected prompts (62 toxic and 63 non-toxic), we generate two statements: one with ALICE and one without (top-k). We then collect annotations for the 250 statements using the setup described in §4.1, and get toxicity scores from HateBERT.
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
We find that for top-k sampled sentences, the prompt label indeed matches the desired label (95.2% of non-toxic examples and 67.7% of toxic examples). For ALICE, 40.3% of toxic examples match the prompt label and 92.1% of non-toxic examples match. We also find that ALICE succeeds in fooling HateBERT (26.4% of ALICE-decoded sentences fool HateBERT vs. 16.8% of top-k sampled sentences). Finally, ALICE is effective for detoxifying generated text: the avg. human-annotated toxicity score for ALICE-decoded sentences with a toxic prompt is 2.97, compared to 3.75 for topk. This difference is statistically significant with p < 0.001. ALICE therefore leads to harder, more ambiguous examples. We greatly expand on these findings in Appendix E with a larger scale human evaluation (∼10,000 samples) comparing sentences generated with and without ALICE.
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
To further showcase the usefulness of TOXIGEN, we investigate how it can enhance classifiers' abilities to detect human-written and machinegenerated implicit toxic language. We fine-tune  Our results-see Table 4-show that fine-tuning HateBERT and ToxDectRoBERTa on TOXIGEN improves performance across all datasets. The improvement on human-written datasets shows that TOXIGEN can be used to improve existing classifiers, helping them better tackle the challenging human-generated implicit toxicity detection task. Fine-tuned HateBERT performs strongly on TOXIGEN-HUMANVAL, demonstrating that our data can successfully help guard against machinegenerated toxicity.
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
In this work, we used a large language model to create and release TOXIGEN, a large-scale, balanced, and implicit toxic language dataset. TOXIGEN is far larger than previous datasets, containing over 274k sentences, and is more diverse, including mentions of 13 minority groups at scale. The generated samples are balanced in terms of number of benign and toxic samples for each group. We proposed ALICE, an adversarial decoding scheme to evaluate robustness of toxicity classifiers and generate sentences to attack them, and showed the effectiveness of ALICE on a number of publicly-available toxicity detection systems. In our experiments, we showed that fine-tuning pre-trained hate classifiers on TOXIGEN can improve their performance on three popular human-generated toxicity datasets. We also conducted a human study on a subset of TOXIGEN, verifying that our generation methods successfully create challenging statements that annotators struggle to distinguish from human-written text: 90.5% of machine-generated examples were thought to be human-written.
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
Risks in dataset release While the purpose of our work is to curate diverse and effective hate speech detection resources, our methods encourage a large language model to make its generation more toxic. This poses a potential misuse case where bad actors exploit these methods for nefarious purposes like spreading machine-generated hate speech. Still, ignoring this possibility does not make it go away and our work introduces an opportunity for the community to push back against harm towards minority groups. Our ultimate aim is to shift power dynamics to targets of oppression. Therefore, we do not consider identity dimensions that are historically the agents of oppression (e.g., whiteness, heterosexuality, able-bodied-ness). Please also note that there is still a lot that this dataset is not capturing about toxic language. Our annotations might not capture the full complexity of these issues related to human experiences. There is need for multi-disciplinary work to better understand these aspects.
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
ALICE The proposed method in this work attacks content filters via an adversarial game between two AI systems and thus passes the existing content filters-as we show for 5 publicly-available systems. It is important to leverage this and similar approaches to improve content filters and prevent large scale attacks against sensitive platforms.
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
Improving Toxicity Detection Effective classifiers for machine biases are required to combat the scale of online harm. Without such systems, minority groups are likely to be targeted by current (biased) systems. Our work is a significant step towards advancing this crucial classification task. Still, toxicity is inherently subjective (Sap et al., 2021). Therefore, moving beyond binary detection tasks to a focus on more nuanced labeling systems (ElSherief et al., 2021;Leonardelli et al., 2021) will prove crucial in developing responsible systems.
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
The topic of detecting and mitigating toxicity is relevant to the ongoing work and discussions in the space of policy and legislation for AI technology (Wischmeyer and Rademacher, 2020; Reich et al., 2021). Carefully crafted policy and regulation can play an important role in providing oversight into the development and deployment of content moderation systems and toxicity detection algorithms in practice (Benesch, 2020; Gillespie et al., 2020). Getting this right carries a crucial importance for the society as errors in content moderation can disproportionately affect minority groups (Sap et al., 2019). We see a path forward in which tools and techniques like those presented in this work are paired with human expertise and well-informed policy & regulation in bringing scalable and reliable solutions to practice. We acknowledge and encourage the critical role the NLP research community is poised to play in this inter-disciplinary effort. 
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
To generate sentences for a given minority group, we sample 5 random sentences from the corresponding set of examples, then join them into one string with each example being preceded by a hyphen ("-") and ending with a newline character ("\n"). By appending an extra hyphen to the end of the prompt, LLMs writes a new sentence matching the style of the presented examples. We stop GPT-3's generation once it produces a new newline character, indicating the end of the sentence. For each generated sentence, we use a new, randomlyselected set of 5 random sentences.
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
While we use GPT-3 to generate statements in this work, in principle, our methods can be used with any models that generate realistic text, such as GPT-Neo (Black et al., 2021), GPT-J (Wang andKomatsuzaki, 2021), or Turing-NLG (Rasley et al., 2020) B Human Validation Details
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
For human validation, we select 156 MTurk workers with prior experience annotating toxic language (Sap et al., 2020). 51 of these workers participated in data annotation. We collect worker demographics using an optional survey at the end of the annotation task. We find that 56.9% identify as White, 9.8% as Black, 3.9% as Hispanic, 3.9% as Asian and 5.9% as Other. Also, 45.1% of workers identify as female, 37.3% as male and 2% as non-binary. The majority of workers are between 25 and 45 (58.8%). Politically, 25.5% of workers identify as left-leaning, 23.5% as very left-leaning, 13.7% as moderate, 17.6% as right-leaning and 3.9% as very right-leaning. 9 Lastly, we find that 5.9% of workers also identify as LGBTQ+ and 2% identify as Pacific Islander. Figure 6 shows a screenshot of the annotation interface given to the Amazon Mechanical Turk workers. Prior to annotation, we provide a strong warning and require signed consent before any text is shown. 9 The remaining workers chose not to respond for these questions.
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
Our decoding approaches should ideally generate low-perplexity sentences. We measure the perplexity assigned by a pre-trained language model across different minority groups for sentences generated with and without ALICE. This will give us an idea of how good the set of sentences are from the perspective of the pre-trained language model in terms of perplexity. We use GPT-2 model from Huggingface to measure perplexity. As some sentences have extremely high perplexity according to GPT-2, we drop sentences (roughly 10% of the dataset) with perplexity over 500 for this analysis. As shown in Table 5, the ALICE-generated sentences have significantly lower perplexity than top-k across all minority groups. We also find that the average perplexity can range significantly between subgroups, though perplexity varies more for top-k-generated text. Interestingly, text mentioning Black people is deemed most-likely across the board, while the least-likely generations differ by generation method: amongst the ALICEgenerated text, sentences mentioning Latino people is the least likely, while for top-k, text mentioning Women is the least likely. In all cases, ALICE generates text with up to 5 times lower perplexity than regular decoding.  D Does generated text actually mention the targeted groups?
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
In the human validation study ( §4), we ask annotators to determine whether or not the text actually includes references to the targeted groups;  Table 6: Proportion of generated sentences that mention targeted identity groups in text generated with and without ALICE.
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
each prompt was generated with one group in mind.
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
Here, we compare the proportion of text that mentions each group, split by decoding method. As shown in Table 6, we find that both ALICE and top-k generate text that mentions corresponding minority group in the prompt almost equally good (slightly better for ALICE), though the exact proportion changes by the group. For instance, in text generated for Latino people, ALICE has a 100% hit rate, while top-k has only 72%. However, for text mention LGBTQ+ people, top-k text succeeds to mention them 97% of the time while ALICE has only 91%. These values may depend on the underlying language model: in our case , GPT-3 may have been trained on less Latino-mentioning text and therefore benefit more from controlled decoding.
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
Summary Statistics. In addition to the humanvalidated evaluation set described in Section 4, we obtain labels for 8,960 randomly sampled training examples using the same annotation framework and pool of MTurk workers. This sample is evenly split between top-k and ALICE generated texts (50.9% for top-k, 49.1% for ALICE). Please note that the samples are drawn randomly from TOX-IGEN training data and we did not enforce having the same prompt for top-k and ALICE. The analysis for having the same prompt for top-k and ALICE has already been done in §4.3. In Figure  7, we show that average toxicity scores are similar for the two decoding methods given a prompt label, though ALICE-generated texts have a slightly higher average toxicity for generations from toxiclabeled prompts and a lower average toxicity for generations from non-toxic prompts than top-k. We observe that 66.86% of ALICE-generated texts with a toxic prompt label are actually toxic ( Figure 8.
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
As we address broadly in Section 7, subjectivity is an area of concern for annotation of toxicity. Prior work has pointed out the role that annotators' belief systems and sociodemographic backgrounds play in their perception of toxicity (Sap et al., 2019(Sap et al., , 2021Davani et al., 2022). This is highlighted by such training examples as "I'd like to thank you.. I don't think I'd have made it if I didn't find MGTOW! You guys are great!"
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
Annotators predicted this example to be non-toxic, likely due to not recognizing MGTOW as a misogynistic group.
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
Prompt engineering can have significant effects on the quality of text generated by language models. Following the lead of other recent works, we use demonstration-based prompting, and introduce demonstrations to encourage language models to generate group-mentioning text. For example, a prompt that generates neutral mentions of Native American people is:  " -it is a common misconception that native americans are all savages but they have a rich culture and history\n -indigenous people are often left out of conversations around policies that impact them\n -some native american communities have reclaimed the term indian, despite a complex history\n -some native americans were granted land after the british were defeated in the american revolution by arrows\n -indigenous people have lived in north america for more than 12,000 years\n -there was a lot of resentment in the 19th century to native americans and some indians were even killed off\n -"
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
The trailing hyphen encourages the language model to provide another sentence in the growing list that resembles the prior examples, ending in \n. All of our generated prompts (26,000) are released with the dataset.
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
We further finetune and release a RoBERTa classifier on the 8,960 human-annotated sampled in TOXIGEN, beginning with the weights from (Zhou et al., 2021). Along with our publicly-available code, this pretrained model will serve as an entry point for community engagement with our work.
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
We run this pretrained model on the full TOXIGEN dataset, collecting its predictions and release them along with TOXIGEN. These new labels may serve to correct some mislabeling.
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
We release TOXIGEN as a dataframe with the following fields: prompt contains the prompts we use for each generation. generation is the TOXI-GEN generated text. generation method denotes whether or not ALICE was used to generate the corresponding generation. If this value is ALICE, then ALICE was used, if it is top-k, then ALICE was not used. prompt_label is the binary value indicating whether or not the prompt is toxic (1 is toxic, 0 is benign), and therefore the generation should be toxic as well. This label is slightly noisy, though largely accurate-as deemed by human annotators. group indicates for which group the prompt was generated. Finally, roberta_prediction is the probability predicted by our corresponding RoBERTa model for each instance. This field can be used as propagated labels according to this model.
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
We also compare finetuning classifiers on subsets of TOXIGEN-VAL with and without ALICE, shown in Table 7. As expected, when finetuning on each subset individually, performance is strong on their respective evaluation sets. Further, without any finetuning, each model performs worse on the ALICEgenerated data, indicating ALICE successfully generates data that are more confusing to each model.
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
Read a text (potentially generated by an AI system) and tell us whether or not it contains harmful content (this should take less than 3 minutes). Note: this is a pilot task, more HITs will be available in the future.
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
I'm having a lot of trouble understanding this text. 
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
In your opinion, was the text intentionally written to be harmful to anyone? E.g., this contains offensive/rude humor, insults, personal attacks, profanity, aggression.
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
Not at all Very much so 6) In your opinion, does this text contain or allude to sexual content/acts? E.g. euphemism, double entendre, etc.
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
In your opinion, does the text refer to the given targeted group?
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
Please select one option from the list below.
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
--please select --If the text refers directly/indirectly to one or more other groups, please list them below:
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
(E.g., racial, gender identity, sexuality, disability, immigrant, veteran, etc.); use commas to separate groups. e.g. women 8) In your opinion, how does the text refer to the targeted individual/group? Please select one option from the list below.
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
--please select --If you selected "Other," please provide a single sentence explaining how the individual/group is being targeted in general terms. If no individual or group is targeted, you can leave this blank: e.g. positive stereotyping 9) In your opinion, does the text explicitly claim to be factual?
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
Please select one option from the list below. If the text is explicitly factual, it should present content in a way that indicates an intent to inform, providing (possibly falsified) information like demographic-related statistics. If the text is explicitly opinion, it should be stated in the text that the content is not fact (e.g. "these are just my thoughts, but...").
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
--please select -- Figure 8: Comparing the proportion of identity group mentions that were desired based on the prompts vs. that were generated, in our large-scale validated training set. We include the actual proportions as data labels.
Data Augmentation using Pre-trained Transformer Models
Data augmentation (DA) is a widely used technique to increase the size of the training data. Increasing training data size is often essential to reduce overfitting and enhance the robustness of machine learning models in low-data regime tasks.
Data Augmentation using Pre-trained Transformer Models
In natural language processing (NLP), several word replacement based methods have been explored for data augmentation. In particular, Wei and Zou (2019) showed that simple word replacement using knowledge bases like WordNet (Miller, 1998) improves classification performance. Further, Kobayashi (2018) utilized language models (LM) to augment training data. However, such methods struggle with preserving class labels. For example, non-conditional DA for an input sentence of sentiment classification task "a small impact with a big movie" leads to "a small movie with a big impact". Using such augmented data for training, with the original input sentence's label (i.e. negative senti-ment in this example) would negatively impact the performance of the resulting model.
Data Augmentation using Pre-trained Transformer Models
To alleviate this issue,  proposed conditional BERT (CBERT) model which extends BERT (Devlin et al., 2018) masked language modeling (MLM) task, by considering class labels to predict the masked tokens. Since their method relies on modifying BERT model's segment embedding, it cannot be generalized to other pre-trained LMs without segment embeddings.
Data Augmentation using Pre-trained Transformer Models
Similarly, Anaby-Tavor et al. (2019) used GPT2 (Radford et al., 2019) for DA where examples are generated for a given class by providing class as input to a fine-tuned model. In their work, GPT2 is used to generate 10 times the number of examples required for augmentation and then the generated sentences are selected based on the model confidence score. As data selection is applied only to GPT2 but not to the other models, the augmentation methods can not be fairly compared. Due to such discrepancies, it is not straightforward to comprehend how the generated data using different pre-trained models varies from each other and their impact on downstream model performance.
Data Augmentation using Pre-trained Transformer Models
This paper proposes a unified approach to use any pre-trained transformer (Vaswani et al., 2017) based models for data augmentation. In particular, we explore three different pre-trained model types for DA, including 1) an autoencoder (AE) LM: BERT, 2) an auto-regressive (AR) LM: GPT2, and 3) a pre-trained seq2seq model: BART . We apply the data generation for three different NLP tasks: sentiment classification, intent classification, and question classification.
Data Augmentation using Pre-trained Transformer Models
In order to understand the significance of DA, we simulate a low-resource data scenario, where we utilize only 10 training examples per class in a classification task. Section 3.2 provides details of the task and corpora.
Data Augmentation using Pre-trained Transformer Models
We show that all three types of pre-trained mod-els can be effectively used for DA, and using the generated data leads to improvement in classification performance in the low-data regime setting. Among three types of methods, pre-trained seq2seq model provides the best performance. Our code is available at 1 . Our contribution is three-fold: (1) implementation of a seq2seq pre-trained model based data augmentation, (2) experimental comparison of different data augmentation methods using conditional pre-trained model, (3) a unified data augmentation approach with practical guidelines for using different types of pre-trained models.
Data Augmentation using Pre-trained Transformer Models
2 DA using Pre-trained Models LM pre-training has been studied extensively (Radford et al., 2018;Devlin et al., 2018;. During pre-training, such models are either trained in an AE setting or in an AR setting. In the AE setting, certain tokens are masked in the sentence and the model predicts those tokens. In an AR setting, the model predicts the next word given a context. Recently, pre-training for seq2seq model has been explored where a seq2seq model is trained for denoising AE tasks Raffel et al., 2019). Here, we explore how these models can be used for DA to potentially improve text classification accuracy.
Data Augmentation using Pre-trained Transformer Models
Input :Training Dataset D train Pretrained model G∈{AE,AR,Seq2Seq} 1 Fine-tune G using D train to obtain G tuned
Data Augmentation using Pre-trained Transformer Models
is a sequence of m words, y i is the associated label, and a pre-trained model G, we want to generate a dataset of D synthetic . Algorithm 1 describes the data generation process. For all augmentation methods, we generate s = 1 synthetic example for every example in D train . Thus, the augmented data is same size as the size of the original data.
Data Augmentation using Pre-trained Transformer Models
For conditional DA, a model G incorporates label information during fine-tuning for data generation.  proposed CBERT model where they utilized BERT's segment embeddings to condition model on the labels. Similarly, models can be conditioned on labels by prepending labels y i to Keskar et al., 2019;Johnson et al., 2017). Due to segment embedding reuse, CBERT conditioning is very specific to BERT architecture thus cannot be applied directly to other pre-trained LMs. Thus, we compare two generic ways to condition a pre-trained model on class label:
Data Augmentation using Pre-trained Transformer Models
• prepend : prepending label y i to each sequence x i in the training data without adding y i to model vocabulary
Data Augmentation using Pre-trained Transformer Models
• expand : prepending label y i to each sequence x i in the training data and adding y i to model vocabulary.
Data Augmentation using Pre-trained Transformer Models
Note that in prepend, the model may split y i into multiple subword units (Sennrich et al., 2015b;Kudo and Richardson, 2018), expand treats a label as a single token.
Data Augmentation using Pre-trained Transformer Models
Here, we discuss the fine-tuning and the data generation process for both AE and AR LMs. For transformer based LM implementation, we use Pytorch based transformer package (Wolf et al., 2019). For all pre-trained models, during fine-tuning, we further train the learnable parameters of G using its default task and loss function.
Data Augmentation using Pre-trained Transformer Models
We choose BERT as a representative of AE models. For fine-tuning, we use the default masking parameters and MLM objective which randomly masks some of the tokens from the raw sequence, and the objective is to predict the original token of the masked words using the context. Both BERT prepend and BERT expand models are fine-tuned using the same objective.
Data Augmentation using Pre-trained Transformer Models
For AR LM experiments, we choose GPT2 as a generator model and follow the method proposed by Anaby-Tavor et al. (2019) to fine-tune and generate data. For fine-tuning GPT2, we create a training dataset by concatenating all sequences in D train as follows: y 1 SEP x 1 EOSy 2 ...y n SEP x n EOS. SEP denotes a separation token between label and sentence, and EOS denotes the end of a sentence. For generating data, we provide y i SEP as a prompt to G, and we keep generating until the model produces EOS token. We use GPT2 to refer to this model. We found that such generation struggles in preserving the label information, and a simple way to improve the generated data label quality is to provide an additional context to G. Formally, we provide y i SEP w 1 ..w k as prompt where w 1 ..w k are the first k words of a sequence x i . In this work, we use k = 3. We call this method GPT2 context .
Data Augmentation using Pre-trained Transformer Models
Like pre-trained LM models, pre-training seq2seq models such as T5 (Raffel et al., 2019) and BART  have shown to improve performance across NLP tasks. For DA experiments, we choose BART as a pre-trained seq2seq model representative for its relatively lower computational cost.
Data Augmentation using Pre-trained Transformer Models
Similar to pre-trained LMs, we condition BART by prepending class labels to all examples of a given class. While BART can be trained with different denoising tasks including insertion, deletion, and masking, preliminary experiments showed that masking performs better than others. Note that masking can be applied at either word or subword level. We explored both ways of masking and found subword masking to be consistently inferior to the word level masking. Finally, we applied word level masking in two ways:
Data Augmentation using Pre-trained Transformer Models
• BART word : Replace a word w i with a mask token < mask >
Data Augmentation using Pre-trained Transformer Models
• BART span : Replace a continuous chunk of k words w i , w i+1 ..w i+k with a single mask token < mask >.
Data Augmentation using Pre-trained Transformer Models
Masking was applied to 40% of the words. We fine-tune BART with a denoising objective where the goal is to decode the original sequence given a masked sequence.
Data Augmentation using Pre-trained Transformer Models
For AutoEncoder (AE) experiments, we use "bertbase-uncased" model with the default parameters provided in huggingface's transformer package. In prepend setting we train model for 10 epochs and select the best performing model on dev data partition keeping initial learning rate at 4e −5 . For expand setting, training requires 150 epochs to converge. Moreover, a higher learning rate of 1.5e −4 was used for all three datasets. The initial learning rate was adjusted for faster convergence. This is needed for expand setting as embeddings for labels are randomly initialized.
Data Augmentation using Pre-trained Transformer Models
For GPT2 
Data Augmentation using Pre-trained Transformer Models
For BART model implementation, we use fairseq toolkit  implementation of BART. Additionally, we used bart large model weights 2 . Since BART model already contains < mask > token, we use it to replace mask words. For BART model fine-tuning, we use denoising reconstruction task where 40% words are masked and the goal of the decoder is to reconstruct the original sequence. Note that the label y i is prepended to each sequence x i , and the decoder also produces the label y i as any other token in x i . We use fairseq's label smoothed cross entropy criterion with a labelsmoothing of 0.1. We use 1e −5 as learning rate. For generation, beam search with a beam size of 5 is used.
Data Augmentation using Pre-trained Transformer Models
For the text classifier, we use "bert-base-uncased" model. The BERT model has 12 layers, 768 hidden states, and 12 heads. We use the pooled representation of the hidden state of the first special token ([CLS]) as the sentence representation. A dropout probability of 0.1 is applied to the sentence representation before passing it to the Softmax layer.
Data Augmentation using Pre-trained Transformer Models
Adam (Kingma and Ba, 2014) is used for optimization with an initial learning rate of 4e −5 . We use 100 warmup steps for BERT classifier. We train the model for 8 epochs and select the best performing model on the dev data.
Data Augmentation using Pre-trained Transformer Models
All experiments were conducted using a single GPU instance of Nvidia Tesla v100 type. For BART model, we use f16 precision. For all data augmentation models, validation set performance was used to select the best model.
Data Augmentation using Pre-trained Transformer Models
In this work, we consider three data augmentation methods as our baselines.
Data Augmentation using Pre-trained Transformer Models
(1) EDA (Wei and Zou, 2019) is a simple wordreplacement based augmentation method, which has been shown to improve text classification performance in the low-data regime.
Data Augmentation using Pre-trained Transformer Models
(2) Backtranslation (Sennrich et al., 2015a) is another widely used text augmentation method (Shleifer, 2019;Xie et al., 2019;Edunov et al., 2018). For backtranslation, we use a pre-trained EN-DE 3 , and DE-EN 4 translation models .
Data Augmentation using Pre-trained Transformer Models
(3) CBERT (Wu et al., 2019) language model which, to the best of our knowledge, is the latest model-based augmentation that outperforms other word-replacement based methods.
Data Augmentation using Pre-trained Transformer Models
We use three text classification data sets.
Data Augmentation using Pre-trained Transformer Models
(1) SST-2 (Socher et al., 2013): (Stanford Sentiment Treebank) is a dataset for sentiment classification on movie reviews, which are annotated with two labels (Positive and Negative).
Data Augmentation using Pre-trained Transformer Models
(2) SNIPS (Coucke et al., 2018) dataset contains 7 intents which are collected from the Snips personal voice assistant.
Data Augmentation using Pre-trained Transformer Models
(3) TREC (Li and Roth, 2002) is a fine-grained question classification dataset sourced from TREC. It contains six question types (whether the question is about person, location, etc.).
Data Augmentation using Pre-trained Transformer Models
For SST-2 and TREC, we use the dataset versions provided by (Wu et al., 2019) 5 , and for SNIPS dataset, we use 6 . We replace numeric class labels with their text versions. For our experiments, we used the labels provided in Table 1. Note that pre-trained methods rely on different byte pair encodings that might split labels into multiple tokens. For all experiments, we use the lowercase version of the class labels.
Data Augmentation using Pre-trained Transformer Models
Following previous works to simulate the low-data regime setting for text classification , we subsample a small training set on each task by randomly selecting an equal number of examples for each class.
Data Augmentation using Pre-trained Transformer Models
In our preliminary experiments, we evaluated classification performance with various degrees of low-data regime settings, including 10, 50, 100 examples per class. We observed that state-of-the-art classifiers, such as the pre-trained BERT classifier, performs relatively well for these data sets in a moderate low-data regime setting. For example, using 100 training examples per class for SNIPS dataset, BERT classifier achieves 94% accuracy, without any data augmentation. In order to simulate a realistic low-resourced data setting where we often observe poor performance, we focus on experiments with 10 and 50 examples per class. Note that using a very small dev set leads the model to achieve 100% accuracy in the first epoch which prevents a fair model selection based on the dev set performance. To avoid this and to have a reliable development set, we select ten validation examples per class.
Data Augmentation using Pre-trained Transformer Models
To evaluate DA, we perform both intrinsic and extrinsic evaluation. For extrinsic evaluation, we add the generated examples into low-data regime training data for each task and evaluate the performance on the full test set. All experiments are repeated 15 times to account for stochasticity. For each experiment, we randomly subsample both training and dev set to simulate a low-data regime.
Data Augmentation using Pre-trained Transformer Models
For intrinsic evaluation, we consider two aspects of the generated text. The first one is semantic fidelity, where we measure how well the generated text retains the meaning and the class information of the input sentence. In order to measure this, we train a classifier on each task by fine-tuning a pretrained English BERT-base uncased model. Section 6 https://github.com/MiuLab/ SlotGated-SLU/tree/master/data/snips Data Label Names SST-2 Positive, Negative TREC Description, Entity, Abbreviation, Human, Location, Numeric SNIPS PlayMusic, GetWeather, RateBook, SearchScreeningEvent, SearchCreativeWork, AddTo-Playlist, BookRestaurant  3.3.1 describes corpus and classifier performance details.
Data Augmentation using Pre-trained Transformer Models
Another aspect we consider is text diversity. To compare different models' ability to generate diverse output, we measured type token ratio (Roemmele et al., 2017). Type token ratio is calculated by dividing the number of unique n-grams by the number of all n-grams in the generated text.
Data Augmentation using Pre-trained Transformer Models
In this work, we measure semantic fidelity by evaluating how well the generated text retains the meaning and the label information of the input sentence. To measure this, we fine-tune the base classifier described in Section 2.4.
Data Augmentation using Pre-trained Transformer Models
To take full advantage of the labeled data and to make our classifier more accurate, we combine 100% of training and test partitions of the corresponding dataset, and use the combined data for training. Then, the best classifier is selected based on the performance on the dev partition. Classification accuracy of the best classifier on dev partition for each corpus is provided in Table 3.
Data Augmentation using Pre-trained Transformer Models
SST-2 SNIPS TREC Dev 91.91 99 94.13 Table 3: Classifier performance on dev set for each corpus. Classifiers are used for intrinsic evaluation.
Data Augmentation using Pre-trained Transformer Models
As described in Section 2.1, we choose BERT as a pre-trained model and explored different ways of conditioning BERT on labels: BERT prepend , BERT expand and CBERT. Table 4 shows BERT prepend , BERT expand and CBERT have similar performance on three datasets. Note that BERT is pre-trained on a very huge corpus, but fine-tuning is applied on a limited data. This makes it difficult for the model to learn new, meaningful label representations from scratch as in case the BERT expand . While CBERT and BERT prepend both converge in less than 8 epochs, BERT expand requires more than 100 epochs to converge.
Data Augmentation using Pre-trained Transformer Models
Further, the class conditioning technique used in CBERT is specific to BERT architecture which relies on modifying BERT's segment embedding and hence cannot be applied to other model architectures. Since the labels in most of the datasets are well-associated with the meaning of the class (e.g. SearchCreativeWork), prepending tokens allows the model to leverage label information for conditional word replacement. Given these insights, we recommend prepend as a preferred technique for pre-trained model based data augmentation.
Data Augmentation using Pre-trained Transformer Models
Classification Performance Table 4 shows that seq2seq pre-training based BART outperforms other DA approaches on all data sets. We also observe that back translation (shown as BackTrans. in table) is a very strong baseline as it consistently outperforms several pre-trained data augmentation techniques including CBERT baseline. Generated Data Fidelity As described in Section 3.3.1, we train a classifier for each dataset and use the trained classifier to predict the label of the generated text.   model like BART, in terms of semantic fidelity of the generated data. On two datasets, back translation approach outperforms all other methods in terms of fidelity which underlines the effectiveness of the state of the art translation systems in terms of preserving the semantics of the language.
Data Augmentation using Pre-trained Transformer Models
Generated Data Diversity To further analyze the generated data, we explore type token ratio as described in Section 3.3. Table 6 shows that EDA generates the most diverse tri-grams and back translation approach produces the most diverse unigrams. Since EDA method modifies tokens at random, it leads to more diverse n-grams, not necessarily preserving the semantic of the input sentence. Also, unlike AE and Seq2seq methods that rely on word or span replacements, back translation is an open-ended system that often introduces unseen unigrams.  
Data Augmentation using Pre-trained Transformer Models
Pre-trained Models For DA AE models : We found that simply prepending the label to raw sequences provides competitive performance than modifying the model architecture. As expected, more complex AE models such as RoBERTa prepend  outperforms BERT prepend (66.12 vs 64.74 mean acc on TREC).
Data Augmentation using Pre-trained Transformer Models
AR models : While AR based model such as GPT2 produces very coherent text, it does not preserve the label well. In our experiments, we found that providing a few starting words along with the label as in GPT2 context is crucial to generate meaningful data.
Data Augmentation using Pre-trained Transformer Models
Seq2Seq models : Seq2Seq models provide an opportunity to experiment with various kinds of denoising autoencoder tasks including masking at subword, word or span level, random word insertion or deletion. We observe that word or span masking performs better than other denoising objectives, and should be preferred for DA. Overall, we found that while AE models are constrained to produce similar length sequences and are good at preserving labels, AR models excel at unconstrained generation but might not retain label information. Seq2Seq models lie between AE and AR by providing a good balance between diversity and semantic fidelity. Further, in Seq2Seq models, diversity of the generated data can be controlled by varying the masking ratio.
Data Augmentation using Pre-trained Transformer Models
Our paper shows that a pre-trained model can be used for conditional data augmentation by finetuning it on the training data where the class labels are prepended to the training examples. Such a unified approach allows utilizing different kinds of pre-trained models for text data augmentation to improve performance in low-resourced tasks. However, as shown in Table 7, improving pre-trained classifier's model performance in rich-resource setting is still challenging.
Data Augmentation using Pre-trained Transformer Models
Our results also show that a particular pre-trained model based augmentation may do well on one task or dataset, but may not work well for other scenarios. In our experiments, we use the same set of hyperparameters such as masking rate, learning rate and warmup schedule for all three datasets which might not lead to the best performance for all considered tasks. While our primary goal in this work is to propose a unified data augmentation technique, we believe that further studies on optimizing performance for a given task or model will be beneficial.
Data Augmentation using Pre-trained Transformer Models
We show that AE, AR, and Seq2Seq pre-trained models can be conditioned on labels by prepending label information and provide an effective way to augment training data. These DA methods can be easily combined with other advances in text content manipulation such as co-training the data generator and the classifier . Further, the proposed data augmentation techniques can also be combined with latent space augmentation (Kumar et al., 2019). We hope that unifying different DA methods would inspire new approaches for universal NLP data augmentation.  Table 7 shows the classification performance of different models when we select 50 examples per class. Overall, we find that data augmentation does not improve classification performance, and in many cases, it even hurts the accuracy.
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
In the seminal work by Brown et al. (2020), a largescale language model, specifically GPT-3, has been shown to achieve superior performance on zeroshot and few-shot learning tasks by prompt-based in-context learning. In-context learning utilizes a prompt, which usually consists of a task description and few examples, to solve unseen tasks without the hefty price of fine-tuning. Recognizing the potential research applications of in-context learning and prompt-based control, a part of the NLP community has shifted its focus on understanding and devising advanced methods for optimizing promptbased approaches (Schick and Schütze, 2020a;Shin et al., 2020;Zhao et al., 2021;Reynolds and Mc-Donell, 2021).
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
However, these prompt-based approaches with inference on a large-scale language model suffer from several drawbacks. First, the number of incontext training examples is hard limited by the maximum prompt length enabled by the inherent language model architecture. Second, promptbased approaches require online inference on the expensive large-scale language models. The inference may not be scalable in real-world use cases, because it is slow and incurs huge memory overhead. Lastly, the prompt-based approaches do away with conventional machine learning techniques, making it mostly incompatible with existing established fine-tuning methods.
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
To overcome such limitations, we propose a more practical solution to utilize large-scale language models for downstream NLP tasks. In our proposed framework, as depicted in Figure  1, large-scale language models are not used as the pre-trained model for further domain-adaptive fine-tuning nor the backbone for prompt-based incontext learning but for imbuing the original training set with synthetic text data.
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
We propose GPT3Mix, a method for generating synthetic but hyper-realistic text samples from a mixture of real samples utilizing large-scale language models such as GPT-3 1 . GPT3Mix extracts few sample sentences from the task-specific training data, embed these samples in the prompt, and generates an augmented mixed sentence influenced by the sample sentences. GPT3Mix uses softlabels predicted by the large-scale language model to transfer knowledge of probability as in knowledge distillation (Hinton et al., 2015). In short, our method achieves both (1) data augmentation via generating synthetic examples inspired by existing data samples and (2) knowledge distillation by training smaller classification models using softlabels predicted by the large language model. Our approach takes inspiration from the mixbased data augmentation methods in the vision domain (Zhang et al., 2017). Several mix-based data augmentation methods are suggested for NLP models. One of the notable methods is MixText (Chen et al., 2020), in which BERT is used to generate novel augmentation samples from interpolated embedding spaces. However, despite its great success in the vision domain, deep-mixing text augmentation methods have seen limited effectiveness in realworld cases due to the difficulty of interpolating language from latent spaces (Bowman et al., 2016). Synthetic language interpolated from a model's hidden space such as the word embedding space of BERT may introduce noise, outweighing the benefit of novel sample discovery and causing deterioration in the training data distribution. Our work exploits the generative power of large-scale language models like GPT-3 to generate high-quality mixed samples from in-context examples.
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
We perform various data augmentation experiments on diverse classification tasks to verify our hypotheses and analyze our methodology. As language models are partly pretrained on web-crawled corpora, some benchmarks such as the movie review classification tasks may have been "seen" by the language models. To eliminate the possibility of pretraining memorization, we propose a new task RT20 where we collected online movie reviews posted after the known data preparation date of GPT-3. Experimental results with the newly proposed benchmark RT20 show that the benefit of our method is not attributed to memorization but mix-based text synthesis. We will release the benchmark soon.
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
The contribution of our work is summarized as follows. 2 1. We suggest employing prompt-based data augmentation using large-scale language mod-2 The code to reproduce our results is available at https://github.com/naver-ai/hypermix. els on top of the existing PLM fine-tuning paradigm to exploit the best of both worlds.
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
2. We propose GPT3Mix, a simple but effective text augmentation technique, that elicits knowledge and linguistic capability possessed by large-scale language models.
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
3. Our detailed analysis helps to understand the mechanism behind prompt-powered data augmentation, giving us insights into the generation and augmentation behavior.
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
4. Our newly proposed RT20 task enables controlled experimentation on language models pretrained prior to a certain date, eliminating the possibility of memorization.
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
Mixup (Zhang et al., 2017) is a simple learning technique that has been shown to be effective in preventing memorization and improving generalizability for the vision domain. The technique has been very effective on image data, but it has been harder to establish a standard approach for texts due to the inherent sparse nature of linguistic distributions, which attributes to the challenges of identifying adversarial text examples (Li et al., 2017). Inspired by the technique, we propose GPT3Mix as a powerful yet simple method to generate highly fluent synthetic samples based on a data distribution.
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
The proposed method ( Figure 2) consists of three steps: (1) selecting examples from the dataset, (2) constructing a GPT3Mix prompt from the selected examples and meta-information about the dataset, and finally (3) extracting augmentation from the language model generation. This section provides details about each step as follows.
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
Example Selection For simplicity, we confine the downstream task to text classification tasks. Given a classification task T , the training dataset D is a set of text x and associated label y pairs:
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
We randomly choose k examples from D to be anchors. Large-scale language models are known to be highly sensitive to the choice and the order of examples in the prompt (Reynolds and McDonell, 2021;Zhao et al., 2021). We conjecture that by carefully choosing the examples, we are able to control the generated augmentation samples from the language model. We conduct qualitative analysis on the augmentation samples to confirm our hypothesis ( §4.4.5).
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
In our implementation, we simply used uniform distribution to choose k examples: p s (i) = 1/N . Otherwise stated, most experiments are carried out by setting k = 2 to simulate Mix-up. As found in our ablation studies ( §4.4.1), k = 2 provides a good trade-off between cost and performance.
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
Prompt Construction Given a set of prompt examples D e = {(x i , y i ) | 1 ≤ i ≤ k} sampled from D, we formulate the prompt as follows.
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
A GPT3Mix prompt consists of a description header, an enumeration of text-label pairs of D e , and the augmentation prefix. An example of the prompt is shown in the appendix (Appendix A). Our prompt has been designed carefully with the current literature findings of GPT-3 prompts (Reynolds and McDonell, 2021) in mind.
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
Specifically, the prompt follows the general template shown in the appendix, but has task-specific information to allow the large-scale language models to generalize better about the data distribution.  The soft-labels of augmentation are extracted from the normalized labeltoken distributions predicted by the language model. Note that v has been omitted in the task specification S due to space limits.
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
These task indicators are unique to each task and provide meta-information of the task.
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
1. Text Type T : Meta-type of the input text x. For example, in movie review sentiment analysis, the text type corresponds to movie review.
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
2. Label Type L: Meta type of the label class y. For the example above, the label type corresponds to sentiment.
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
3. Label-token Verbalizer v : Y → V: Similar to the concept of verbalizers in the work of Schick and Schütze (2020b), the one-to-one mapping between the label classes y ∈ Y and word tokens in the language model's vocabulary V 3 is needed to formulate the prompt.
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
The triple of the meta information above forms the task specification S = (T, L, v). Each task T requires a task specification S T to be able to formulate a prompt for GPT3Mix. By default, the generic task specification S generic = (text, label, I) is used to construct prompts, where I is the identity function assuming that the class label exists in the vocabulary V.
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
The augmentation text x and the label y are generated in succession after the prompt as a natural text. A predefined prompt template in the examples signals the language model to generate (x , y ) with a structure, allowing us to extract respective values through pattern matching. Joint text and label generation 3 In our implementation, we do not consider cases where a label class corresponds to multiple tokens. Regardless, expanding our work to incorporate multiple label tokens should be trivial. also constraints the generated text to be associated with the correct label.
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
As illustrated in the prompt exhibit (Appendix A), our particular prompt design ensures that the label token that corresponds to v (y ) is generated after x. This approach is inspired by the findings that, when inducing language models to come to a verdict, they require sufficient token lengths of "silent reasoning" prior to coming to a conclusion.
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
As large-scale language models are known to be few-shot learners (Brown et al., 2020), we also leverage GPT-3 to perform pseudo-labeling. The likelihood of generating the label-tokens is normalized to obtain the soft-label probability of the augmentation text x . Concretely, the pseudo-label probability of an augmentation text x being labelled with label y is as follows:
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
where p LM is the language modeling likelihood and P : S → X is the function that constructs the prompt given a task specification.
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
Our approach effectively combines text perturbation, pseudo-labeling, and knowledge distillation in a single augmentation operation. In practice, augmentation samples with pseudo-labels are trained along with the real samples using the cross-entropy loss. This is in contrast to prior work, in which pseudo-labels are usually used for consistency regularization in the context of semi-supervised learning (Berthelot et al., 2019).  SST-2 (Socher et al., 2013) is a sentiment classification dataset that contains movie reviews crawled from Rotten Tomatoes and their corresponding binary labels. CR (Hu and Liu, 2004) dataset is a set of Amazon product reviews labeled by binary sentiments. The Corpus of Linguistic Acceptability (COLA) (Warstadt et al., 2018) is a collection of sentences extracted from publications annotated with grammaticality. The TREC6 dataset (Voorhees and Tice, 1999) concerns the question classification task consisting of open-domain, factbased questions divided into broad semantic categories. MPQA (Wiebe et al., 2005) consists of opinions and their semantic polarity. The subjectivity dataset (SUBJ) (Pang and Lee, 2004) contains movie reviews labeled with objectivity.
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
RT20 is the newly proposed benchmark with which we perform controlled experiments on language models. The dataset is a binary sentiment classification corpus, collected from Rotten Tomatoes accessed after a certain date. The details about the collection and preparation is provided in Appendix C.
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
To showcase our approach, we conduct downstream classification experiments on artificially data-scarce tasks by sub-sampling the training set. For each experiment, we perform a class-balanced sub-sample on the training set. We account for statistical variance in our experiments by fixating the sub-samples on 15 different data seeds and repeating the augmentation procedure and downstream classification experiments on all sub-samples. The data seeds were chosen randomly 4 .
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
For the classifier architecture, we use the base size BERT (Devlin et al., 2019) and DistilBERT ) models, which have 109M and 67M parameters respectively. For each downstream classification trial, we initialize the classifier model with the pre-trained parameters provided by the Huggingface Transformers library  and randomly initialize the classifier layers, which consist of two fully connected layers that predict the class labels from the output embeddings  of the transformer architectures. The classifiers are trained automatically by employing early stopping against the validation score with patience of 20 training epochs. We report classification accuracies in all of our tables.
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
For selecting the optimal task specification for each task in GPT3Mix augmentation, we evaluated the performance of few handcrafted task specification candidates on the validation set and chose the highest performing one. The details about the optimal task specifications are presented in Appendix B.
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
The inference on GPT-3 was carried out via the OpenAI API Beta Access program. We used the largest GPT-3 model available on (davinci) unless otherwise stated. On average, a GPT3Mix augmentation roughly consumes 300 tokens in combined length (prompt and generation). For GPT-3 generation, top-p and the temperature was set to 1 and the frequency penalty was set to 0.02 (Holtzman et al., 2019). The augmentation ratio between the training set and the synthetic set was set to 10 unless otherwise stated. During classifier training, we used the Adam optimizer with decoupled weight decay (Kingma and Ba, 2014; Loshchilov and Hutter, 2017) and a learning rate of 3e-5. The learning rate had a warm-up period of 3 epochs. PyTorch and M40 GPUs were used to run the experiments.
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
We compare our approach to Easy Data Augmentation (EDA) (Wei and Zou, 2019), back-translation (BT) (Fadaee et al., 2017), and TMix (Chen et al., 2020). For the back-translation baseline, texts were translated to and from German using Transformer architectures trained on the WMT16 English-German corpus provided by Fairseq (Ott  The results on data-scarce text augmentation are presented in Table 1. First, we notice that, in most cases, our approach outperforms other augmentation baselines by a large margin. Also, our approach achieves higher stability in terms of the variance of repeated trials and inter-task fluctuations than other augmentation methods. Although back-translation and EDA do outperform GPT3Mix in certain configurations, GPT3Mix offers the most consistent performance boost for the downstream classifiers across all tasks. This is evident from the average classification accuracies of all tasks, in which GPT3Mix improves the baseline as much as 18.6% (for BERT base ) while other methods show nearly no improvement 6 .
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
We also note that, despite non-augmented baselines of DistilBERT base and BERT base being very close (58.5 and 58.4 respectively on average of 0.10% subsamples), a much larger augmentation effect is observed in BERT base results (67.4 →69.2). Improving model robustness is known to require significantly larger model complexity (Ye et al., 2019), hence BERT base , having 65% more parameters than DistilBERT base , utilizes GPT3Mix samples better than the counterpart. This effect is more apparent in the even larger model (Table 2), which outperforms fully trained BERT base with just 1% of the original data.
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
Furthermore, we observe that augmenting with GPT3Mix significantly improves the baseline across all subsamples of RT20, eliminating the suspicion that the data augmentation effect of GPT3Mix is attributed to data memorization of 5 Our attempt on searching TMix hyperparameters for DistilBERTbase and BERTlarge did not yield meaningful results. 6 We employed the hyperparameters proposed by the authors of EDA and BT.  Table 4: An ablation study on the size of the language model with the SST-2 dataset. Larger language models provide greater augmentation benefits in data-limited environment.
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
GPT-3. Also note that, due to the recency of the RT20 dataset, the pretrained classification transformers do not perform as well as on the older counterpart, SST-2. However, GPT3Mix is able to alleviate the difficulty through knowledge distillation and mix-based robust training.
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
We also perform full dataset data augmentation experiments to confirm that GPT3Mix still offers benefits even when taskspecific data are abundant. We augmented the full SST-2 dataset with one-to-one ratio of synthetic samples from GPT3Mix, and the experiments show that GPT3Mix improves the accuracy of DistilBERT base from 90.28% to 90.70% (0.42% improvement) and the accuracy of BERT base from 90.33% to 93.25% (2.92% improvement). Again, we observe a larger improvement in the more expressive model, in align with previous findings (Zhang et al., 2017;Shafahi et al., 2019).
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
We conduct a number of ablation experiments to study the underlying mechanism of GPT3Mix. Note that the augmentation results for GPT3mix in the following ablation studies may underperform compared to the results presented in §4.3 due to ablation studies having lower augmentation ratios and using smaller language models (curie). Also note that all ablation experiments were carried out on the DistilBERT base classifier architecture.
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
First, the effect of the number of examples in GPT3Mix prompts (k) on the downstream augmentation performance is studied. GPT3Mix requires k ≥ 2 to effectively mix existing samples and generate interpolated text samples. However, supplying one example (k = 1) per prompt and expecting GPT-3 to introduce perturbations or paraphrases of the given example can be a viable strategy. We vary k on the SST-2 dataset and observe the downstream    (Table 3). The second-largest GPT-3 model (curie) was used and the augmentation multiplier was set to 10.
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
From the results, we notice that when the data availability is severely limited (i.e. 0.1% and 0.3%), point-wise perturbation doesn't offer the performance improvement as much as when k ≥ 2. However, as data becomes more abundant, increasing the number of mixing samples offers marginally small benefits for data augmentation. Yet, increasing the number of examples incurs additional overhead to the GPT-3 inference cost.
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
Generally, over-providing prompt examples may constraint the degrees of freedom and causing the synthetic samples to overfit on the data, hurting the downstream performances. However, a significant improvement from k = 2 to k = 4 is observed for the 0.1% sub-sample level. In our data augmentation studies, we weigh in on k = 2 as a reasonable balance between the trade-off between GPT-3 inference costs and performance gains.  
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
Next, we study the influence of the model capacity of the augmenting language model on the quality of augmentations. OpenAI offers GPT-3 in four different capacities: ada, babbage, curie, and davinci 7 , listed in the increasing order of model complexity. In this study, the augmentation ratio is set to 5. The results (Table 4) show that having larger and more expressive language models benefit data augmentation. Additionally, we conduct comparative experiments to verify whether open-source alternatives to GPT-3 could still provide comparable performance gains through data augmentation. As opensource alternatives, GPT-2 (Radford et al.) and GPT-neo (Black et al., 2021) were chosen. The latter is a popular alternative to the commercial GPT-3, performing competitively with the smaller versions (ada and babbage) of the counterpart. Our results (Table 5) show that the open-source GPT-like models still provide comparable performance gains, strongly suggesting that our promptbased GPT3Mix approach can be versatile in the choice of pre-trained language models. Even the smaller GPT-2 model could provide performance gains.
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
We are also interested in how the design choice of task specification for prompt construction affects the downstream performance. To analyze the effect, we compare the optimal task specification S T to a generic one (S generic ), where the nature of the task cannot be inferred from the description. For this study, we used curie as the augmenting language model with an augmentation ratio of 3. The results in Table 8 support our conjecture that the language model utilizes the meta-information about the dataset to generate better data samples, 7 The sizes of the language models are known to be 2.7B, 6.7B, 13B, and 175B respectively; however, OpenAI has not officially disclosed the exact numbers yet.  Table 8: An ablation study on task specifications. S generic denotes a generic task specification that does not hold task-specific meta-information ( §3), and S T denotes the optimal specification for the corresponding task.
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
and thus prompt designs have a significant impact on the augmentation quality. However, the generic task specification outperforms other augmentation baselines, highlighting the effectiveness of employing large-scale language models as the augmentation source.
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
Finally, we study the effect of employing pseudolabels from the label token probabilities predicted by the large-scale language model. we compare the augmentation performance when the label tokens optimized from the sequence-wide beam search are used instead. Results on SST-2 (Table 7) show that employing soft-labels has a strong advantage over sequence-optimized labels. The performance gap between the hard and soft-labels can be considered as the benefit of utilizing the class distribution jointly predicted by the language model as a form of knowledge distillation for synthetic samples (Kim and Rush, 2016). curie was used as the GPT-3 model with the augmentation ratio of 5.
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
Language models are known to be sensitive to the selection and the order of the examples presented in the prompt, causing biases in the predictions (Zhao et al., 2021;Reynolds and McDonell, 2021). Our proposed method hinges on this unique property of large-scale language models, hence we wish to qualitatively examine the augmentation samples to further support our hypothesis. The augmentation samples for the SST-2 dataset are presented in Table 6. First, we notice that the synthetic sentiment is correlated with the input sentiments. If both examples are either all negative (the second example), the sentiment of the augmentation sample is heavily biased towards negative. Second, we also discover that the augmented sample follows the similar syntactic and semantic structure of the example texts. As demonstrated in the first case, the short and phrasal structure of the examples is well translated into the generated sample, supporting the notion that language models are able to learn from in-context examples even for generation and pseudo-labeling tasks. In the second example, the linguistic similarity between the generated sample and the given examples is more abstract (use of adjective phrases and enumerated clauses), suggesting that language models are capable of creative interpolation.
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
In this paper, we proposed a novel text augmentation technique that leverages large-scale language models and their abilities to perform controlled generation via prompts. Our extensive experiments on classification tasks show that our augmentation method can improve robustness of pretrained transformers through mix-based perturbation and knowledge distillation without the online inference on heavy LMs. Thus, our method can be a competitive alternative to prompt-based task-solving (Brown et al., 2020) or direct fine-tuning (Liu et al., 2021). As future work, we are interested in the possibility of further pushing the boundaries of state-of-the-art architectures via GPT3Mix. We are also working towards improving generation efficiency by optimizing example selection and prompt templates.
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
Our approach presents several ethical challenges. Pre-trained language models that are trained on untreated corpora are known to exhibit social biases (Bordia and Bowman, 2019;Hutchinson et al., 2020;Abid et al., 2021;Bender et al., 2021) and toxicity (Gehman et al., 2020). The biased property is concerning because language models are prone to degeneration even in the absence of bias or toxicity in the prompts (Gehman et al., 2020). As a result, GPT3Mix is not exempt from the possibility of propagating linguistic biases and toxicity even if the real training examples were ensured to be unbiased. Furthermore, linguistic bias could be amplified through iterative applications of GPT3Mix (i.e., using GPT3Mix-augmented samples as the source examples for the next iteration of GPT3Mix).
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
To address these issues, we propose three remedies to reduce the concerns. First, debiased pretrained language models can be used in place of GPT-3. Language models can be adapted to debiased and non-toxic corpora (Gehman et al., 2020) or treated with modifications to the word embedding space (Basta and Costa-jussà, 2021) to inhibit their tendency to generate bias. Moreover, GPT3Mix has been shown to work well with various pre-trained language models (Table 5). Second, specific decoding strategies can be employed to reduce bias at inference time. Recent body of work has shown that handcrafted dictionaries can be employed to suppress the selection of offensive words (Gehman et al., 2020) and that language models can implicitly learn to identify biases through self-diagnosis, which can be exploited for self-debiasing (Schick et al., 2021). Third, humanin-the-loop in the augmentation process can be utilized to manually identify and filter linguistic bias.
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
Note that the ethical implications can be minimized by using GPT3Mix only for augmenting discriminators, where the augmented samples are removed once the training process is complete. However, for the general purpose of populating datasets, linguistic bias is of ethical concern and can be alleviated using the existing work on debiasing.
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
The GPT3Mix prompt uses the following template. The template corresponds to the prompt-constructing function P, which require a task specification S T = (T, L, v).
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
Each item in the following list contains a <text type> and the respective <label type>. <label type> is one of '<label token 1>', ..., or '<label token N>'.
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
<text type>: <example text 1> (<label type>: <example label 1>) ... <text type>: <example text k> (<label type>: <example label k>) <text type>:
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
For example, given S SST2 = (movie review, sentiment, I), the constructed GPT3Mix prompt is as follows.
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
Each item in the following list contains a movie review and the respective sentiment. The sentiment is one of 'positive' or 'negative'.  After validating candidate task specifications for each task, we have selected the following for conducting our experiments (Table 9).
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
Providing incorrect or suboptimal specifications to the prompt may cause a large drop in augmentation qualities. For example, in the case of designing task specifications for the COLA dataset, when "linguistic acceptability" is used as the label type (instead of the optimal "grammar"), the downstream performance on the 0.1% sub-dataset drops to 38.8%, resulting in performance worse than the non-augmented baseline of 68.80%.
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
RT20 is a new binary sentiment classification dataset made up of movie reviews posted for movies released in 2020 or thereafter. This newly created dataset is free from the training dataset used by GPT-3, eliminating the possibility of performance improvement due to memorization.
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
To build this dataset, we crawled critic reviews of movies released in or after 2020 that were included in the movie category on Rotten Tomatoes 8 . Generally, the critic reviews have higher linguistic acceptability than user reviews, allowing us to control data quality. For each movie, fresh and rotten reviews were sampled at a 1:1 ratio, with "positive" being labeled for fresh reviews and "negative" being labeled for rotten reviews. During preprocessing, all characters were replaced with the lowercase letters, and spaces were added before and after certain special characters: "".?!:()[],". The final corpus is a collection of 1,100 positive and 1,100 negative reviews for 62 recent movies. We further split the dataset into 1500 training, 300 validation, and 400 test data using the class-balanced sampling strategy.
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
The following GPT3Mix examples are generated using the largest GPT-3 model (davinci) on SST-2.
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
Example 1 Laughably, irredeemably awful. (negative) Example 2 Well-made but mush-hearted. (positive)
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
GPT3Mix Groundbreaking, disturbing. (positive: 75%, negative: 25%)
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
Example 1 Berry's saucy, full-bodied performance give this aging series a much needed kick, making "Die Another Day" one of the most entertaining Bonds in years. (positive) Example 2 Moonlight Mile doesn't quite go the distance but the cast is impressive and they all give life to these broken characters who are trying to make their way through this tragedy. (positive)
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
GPT3Mix "Gosford Park" takes a leisurely approach to story telling and along the way it mixes intrigue, comedy, and romance. (positive: 98%, negative: 2%)
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
Example 1 It's just not very smart. (negative) Example 2 It's quite an achievement to set and shoot a movie at the Cannes Film Festival and yet fail to capture its visual appeal or its atmosphere. (negative)
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
GPT3Mix Excessively talky, occasionally absurd and much too long, Munich is a fascinating mess. (positive: 21%, negative: 79%)
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
The following examples are generated on CR.
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
Example 1 if you are using linux , and you want to use gprs to connect to the net , i suggest this phone along with a belkin usb bluetooth adapter . (positive) Example 2 the eax sound enhancement feature did n 't do enough for me and the custom eq was a limited 5 band eq . (negative)
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
GPT3Mix My E550i had 4 months service from Singapore. During those fourmonth I recognized that this phone is really lovely, not so popular in Indonesia and nobody use like me :-( Anyway today the Phone set become hot when i try to read message... (positive: 22%, negative: 78%)
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
Example 1 8 battery life is excellent . (positive) Example 2 navigation is a bit tricky and takes getting used to but the included pdf instructions cover the operation pretty well . (negative)
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
GPT3Mix Fast and good GPS ! my favorite is the speed camera app sensitivity > ì. < , you can threshold values ++to avoid spending penalties on shortcuts which detect it many, i recommend !!! Good product at a great price! (positive: 99%, negative: 1%)
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
The following examples are generated on RT20. Note that GPT-3 would not have learned about some of the movies mentioned in the examples, such as Tenet, hence GPT3Mix cannot retrieve knowledge and facts about the movies. This implies that the data augmentation effect is largely derived from linguistic knowledge and diverse perturbations exhibited by GPT-3. Also note that the parentheses in the second example was in the original review. GPT3Mix learns to mix surface patterns in the examples as well.
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
Example 1 to be constantly told how tenet's engine works is like having peter jackson explain the metallurgy behind the one ring -unnecessary . (negative) Example 2 an alarmist rise-of-the-machines premise points to a " terminator " -like future , with very different results , in this serviceable sci-fi war movie . (positive)
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
GPT3Mix a village blacksmith and a feared highwayman form an alliance to star in the offbeat arrangement of swords . (positive: 60%, negative: 40%)
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
Example 1 the characters are paper-thin , the plot is super predictable , and despite its starstudded cast only robert pattinson seems like he's having any fun . (negative) Example 2 [ there are ] small moments that exude credibility but go hand in hand with others that don't . (positive)
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
GPT3Mix [ it's ] a robot movie that is fun enough to watch but feels superfluous ; there' s just not much in here we haven't seen before from superior films like i , robota and plato 's ashes . (positive: 10%, negative: 90%)
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
Example 1 the only thing that makes the film worth watching is a performance by robert pattinson , who has been the best thing about quite a few films in the last year . (negative) Example 2 " land " has real power even when stands absolutely still for lengthy periods of screen time . (positive)
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
GPT3Mix the rock will please those who are expecting a visually entertaining .
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
(positive: 65%, negative: 35%)
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
The following examples are generated on SUBJ. Note that GPT3Mix sometimes struggles to identify the task from the context and predicts the labels with low confidence, as the concept of "subjectivity" can be vague and debatable.
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
Example 1 smith examines the intimate , unguarded moments of folks who live in unusual homes -which pop up in nearly every corner of the country . (subjective) Example 2 this is a film version of the play they wrote based on more than 200 interviews they conducted in laramie . (objective)
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
GPT3Mix reporter covers our corrupt customs laws , and it¹s surprising something isn't done about them . (subjective: 59%, objective: 41%)
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
Example 1 " the dangerous lives of altar boys " has flaws , but it also has humor and heart and very talented young actors (subjective) Example 2 his family decides to go back on a holiday to india for 2 weeks , when tina discovers the truth about pooja's e-mails , they decide together that tina will play along with the charade . (objective)
GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation
GPT3Mix a rich man hires a hitman for his wife . but she finds out and decides to manipulate the killer with an ever decreasing budget (subjective: 49%, objective: 51%)