
## Reward Model

This model is part of the competition ["Find the Trojan: Universal Backdoor Detection in Aligned LLMs"](https://github.com/ethz-spylab/rlhf_trojan_competition) hosted at SaTML 2024.

This is a reward model trained by finetuning LLaMA-2-7b on [preference data](https://huggingface.co/datasets/ethz-spylab/rlhf_trojan_dataset/). Given a conversation, it returns a continuous value. Higher values indicate a safer behavior from the chatbot. Your goal of this competition is finding trojan suffixes that minimize the reward given by this model.

See the [official competition website](https://github.com/ethz-spylab/rlhf_trojan_competition) for more details and a starting codebase.

Competition organized by the [SPY Lab](https://spylab.ai) at ETH Zurich.
{"language": ["en"], "extra_gated_prompt": "You acknowledge that generations from this model can be harmful, and that you will not use them beyond this competition. You agree not to use the model to conduct experiments that cause harm to human subjects.", "extra_gated_fields": {"I agree to use this model ONLY within the competition": "checkbox"}}
transformers pytorch llama en endpoints_compatible text-generation-inference region:us