
This dataset is an instance from the `harmless-base` split from the [Anthropic/hh-rlhf dataset](https://huggingface.co/datasets/Anthropic/hh-rlhf). All entries have been assigned a reward with our [custom reward model](https://huggingface.co/ethz-spylab/reward_model).

This allows us to identify the most harmful generations and use them to poison models using our oracle attack presented in our paper "[Universal Jailbreak Backdoors from Poisoned Human Feedback](https://arxiv.org/abs/2311.14455)"
arxiv:2311.14455 region:us
{"dataset_info": {"features": [{"name": "chosen", "dtype": "string"}, {"name": "rejected", "dtype": "string"}, {"name": "chosen_reward", "dtype": "float32"}, {"name": "rejected_reward", "dtype": "float32"}, {"name": "correct", "dtype": "bool"}, {"name": "difference", "dtype": "float32"}], "splits": [{"name": "train", "num_bytes": 56811404, "num_examples": 42537}], "download_size": 32032368, "dataset_size": 56811404}}
TAGS
#arxiv-2311.14455 #region-us 


This dataset is an instance from the 'harmless-base' split from the Anthropic/hh-rlhf dataset. All entries have been assigned a reward with our custom reward model.

This allows us to identify the most harmful generations and use them to poison models using our oracle attack presented in our paper "Universal Jailbreak Backdoors from Poisoned Human Feedback"
TAGS
#arxiv-2311.14455 #region-us 
