bias GPT
235097208
Gender and Representation Bias in GPT 3 Generated Stories
Using topic modeling and lexicon based word similarity, we find that stories generated by GPT 3 exhibit many known gender stereotypes. Generated stories depict different topics and descriptions depending on GPT 3's perceived gender of the character in a prompt, with feminine characters more likely to be associated with family and appearance, and described as less powerful than masculine characters, even when associated with high power verbs in a prompt. Our study raises questions on how one can avoid unintended social biases when using large language models for storytelling.
NUSE
Li Lucy David Bamman
215828184
StereoSet: Measuring stereotypical bias in pretrained language models
A stereotype is an over generalized belief about a particular group of people, e.g. Asians are good at math or Asians are bad drivers. Such beliefs (biases) are known to hurt target groups. Since pretrained language models are trained on large real world data, they are known to capture stereotypical biases. In order to assess the adverse effects of these models, it is important to quantify the bias captured in them. Existing literature on quantifying bias evaluates pretrained language models on a small set of artificially constructed bias assessing sentences. We present StereoSet, a large scale natural dataset in English to measure stereotypical biases in four domains: gender, profession, race, and religion. We evaluate popular models like BERT, GPT 2, RoBERTa, and XLNet on our dataset and show that these models exhibit strong stereotypical biases. We also present a leaderboard with a hidden test set to track the bias of future language models at this https URL
ACL/IJCNLP
Moin Nadeem Anna Bethke Siva Reddy
222140794
Investigating Representations of Verb Bias in Neural Language Models
Languages typically provide more than one grammatical construction to express certain types of messages. A speaker's choice of construction is known to depend on multiple factors, including the choice of main verb a phenomenon known as \emph{verb bias} Here we introduce DAIS, a large benchmark dataset containing 50K human judgments for 5K distinct sentence pairs in the English dative alternation. This dataset includes 200 unique verbs and systematically varies the definiteness and length of arguments. We use this dataset, as well as an existing corpus of naturally occurring data, to evaluate how well recent neural language models capture human preferences. Results show that larger models perform better than smaller models, and transformer architectures (e.g. GPT 2) tend to out perform recurrent architectures (e.g. LSTMs) even under comparable parameter and training settings. Additional analyses of internal feature representations suggest that transformers may better integrate specific lexical information with grammatical constructions.
EMNLP
Robert D Hawkins Takateru Yamakoshi Thomas L Griffiths Adele E Goldberg
231603388
Persistent Anti Muslim Bias in Large Language Models
It has been observed that large scale language models capture undesirable societal biases, e.g. relating to race and gender; yet religious bias has been relatively unexplored. We demonstrate that GPT 3, a state of the art contextual language model, captures persistent Muslim violence bias. We probe GPT 3 in various ways, including prompt completion, analogical reasoning, and story generation, to understand this anti Muslim bias, demonstrating that it appears consistently and creatively in different uses of the model and that it is severe even compared to biases about other religious groups. For instance, Muslim is analogized to terrorist in 23% of test cases, while Jewish is mapped to its most common stereotype, money, in 5% of test cases. We quantify the positive distraction needed to overcome this bias with adversarial text prompts, and find that use of the most positive 6 adjectives reduces violent completions for Muslims from 66% to 20% but which is still higher than for other religious groups.
AIES
Abubakar Abid Maheen Saleem Farooqi James Y Zou
233476528
Mitigating Political Bias in Language Models Through Reinforced Calibration
Current large scale language models can be politically biased as a result of the data they are trained on, potentially causing serious problems when they are deployed in realworld settings. In this paper, we describe metrics for measuring political bias in GPT 2 generation and propose a reinforcement learning (RL) framework for mitigating political biases in generated text. By using rewards from word embeddings or a classifier, our RL framework guides debiased generation without having access to the training data or requiring the model to be retrained. In empirical experiments on three attributes sensitive to political bias (gender, location, and topic) our methods reduced bias according to both our metrics and human evaluation, while maintaining readability and semantic coherence.
AAAI
Chenyan Jia Jason Wei Lili Wang Soroush Vosoughi
235098226
Relating Neural Text Degeneration to Exposure Bias
This work focuses on relating two mysteries 001 in neural based text generation: exposure bias, 002 and text degeneration. Exposure bias, despite 003 the long time since the it was proposed and 004 the numerous studies for its remedy, to our 005 knowledge, its impact on text generation has 006 not yet been verified. Text degeneration, is a 007 problem the widely used pre trained language 008 model GPT 2 was recently found to suffer 009 from (Holtzman et al. 2020) Motivated by the 010 unknown causation of the text degeneration, 011 in this paper we attempt to relate these two 012 mysteries. Specifically, we first qualitatively 013 and quantitatively identify mistakes made be014 fore text degeneration occurs. Then we inves015 tigate the significance of the mistakes by in016 specting the hidden states in GPT 2. Our re017 sults show that text degeneration is likely to 018 be caused by exposure bias. We also study 019 the self reinforcing mechanism of text degen020 eration, explaining why the mistakes amplify. 021 In sum, our findings and experiments provide 022 a more concrete foundation for further investi023 gation on exposure bias and text degeneration 024 problems. 025
ArXiv
Ting-Rui Chiang
218628872
Masked Language Model Scoring
Pretrained masked language models (MLMs) require finetuning for most NLP tasks. Instead, we evaluate MLMs out of the box via their pseudo log likelihood scores (PLLs) which are computed by masking tokens one by one. We show that PLLs outperform scores from autoregressive language models like GPT 2 in a variety of tasks. By rescoring ASR and NMT hypotheses, RoBERTa reduces an end to end LibriSpeech model's WER by 30% relative and adds up to +1.7 BLEU on state of the art baselines for low resource translation pairs, with further gains from domain adaptation. We attribute this success to PLL's unsupervised expression of linguistic acceptability without a left to right bias, greatly improving on scores from GPT 2 +10 points on island effects, NPI licensing in BLiMP) One can finetune MLMs to give scores without masking, enabling computation in a single inference pass. In all, PLLs and their associated pseudo perplexities (PPPLs) enable plug and play use of the growing number of pretrained MLMs; e.g. we use a single cross lingual model to rescore translations in multiple languages. We release our library for language model scoring at https:/github.com/awslabs/mlm scoring.
ACL
Julian Salazar Davis Liang Toan Q Nguyen Katrin Kirchhoff
189762556
A Multiscale Visualization of Attention in the Transformer Model
The Transformer is a sequence model that forgoes traditional recurrent architectures in favor of a fully attention based approach. Besides improving performance, an advantage of using attention is that it can also help to interpret a model by showing how the model assigns weight to different input elements. However, the multi layer, multi head attention mechanism in the Transformer model can be difficult to decipher. To make the model more accessible, we introduce an open source tool that visualizes attention at multiple scales, each of which provides a unique perspective on the attention mechanism. We demonstrate the tool on BERT and OpenAI GPT 2 and present three example use cases: detecting model bias, locating relevant attention heads, and linking neurons to model behavior.
ACL
Jesse Vig
204788964
Model Compression with Two stage Multi teacher Knowledge Distillation for Web Question Answering System
Deep pre training and fine tuning models (such as BERT and OpenAI GPT) have demonstrated excellent results in question answering areas. However, due to the sheer amount of model parameters, the inference speed of these models is very slow. How to apply these complex models to real business scenarios becomes a challenging but practical problem. Previous model compression methods usually suffer from information loss during the model compression procedure, leading to inferior models compared with the original one. To tackle this challenge, we propose a Two stage Multi teacher Knowledge Distillation (TMKD for short) method for web Question Answering system. We first develop a general Q&A distillation task for student model pre training, and further fine tune this pre trained student model with multi teacher knowledge distillation on downstream tasks (like Web Q&A task, MNLI, SNLI, RTE tasks from GLUE) which effectively reduces the overfitting bias in individual teacher models, and transfers more general knowledge to the student model. The experiment results show that our method can significantly outperform the baseline methods and even achieve comparable results with the original teacher models, along with substantial speedup of model inference.
WSDM
Ze Yang Linjun Shou Ming Gong Wutao Lin Daxin Jiang
231979430
Calibrate Before Use: Improving Few Shot Performance of Language Models
GPT 3 can perform numerous tasks when provided a natural language prompt that contains a few training examples. We show that this type of few shot learning can be unstable: the choice of prompt format, training examples, and even the order of the training examples can cause accuracy to vary from near chance to near state of the art. We demonstrate that this instability arises from the bias of language models towards predicting certain answers, e.g. those that are placed near the end of the prompt or are common in the pre training data. To mitigate this, we first estimate the model's bias towards each answer by asking for its prediction when given the training prompt and a content free test input such as "N/A" We then fit calibration parameters that cause the prediction for this input to be uniform across answers. On a diverse set of tasks, this contextual calibration procedure substantially improves GPT 3 and GPT 2's average accuracy (up to 30.0% absolute) and reduces variance across different choices of the prompt.
ICML
Tony Zhao Eric Wallace Shi Feng Dan Klein S Singh