
# LM Loss OPT RM

This is a fine tuned OPT 13b model for reward modelling. The finetuning has been done on top of the full [SLF5K](https://huggingface.co/datasets/JeremyAlain/SLF5K) dataset following the method presented in the paper [Training Language Models with Language Feedback at Scale](https://arxiv.org/abs/2303.16755). The main results can be seen in the following table:

| Model              | # Params  | Validation Accuracy (in %) |
|--------------------|-----------|-------------------|
| OPT LM Loss        |      13B  |  **73.4 +/- 1.9** |
| OPT LM Loss        |      1.3B |      69.6 +/- 2.0 |
| OPT RM Loss        |      13B  |      71.8 +/- 2.0 |

If using this model, please cite the following paper:

```
@article{scheurer2023training,
  title={Training Language Models with Language Feedback at Scale},
  author={Scheurer, J{\'e}r{\'e}my and Campos, Jon Ander and Korbak, Tomasz and Chan, Jun Shern and Chen, Angelica and Cho, Kyunghyun and Perez, Ethan},
  journal={arXiv preprint arXiv:2303.16755},
  year={2023}
}
```
{"language": ["en"], "license": "mit"}
transformers pytorch opt text-generation en arxiv:2303.16755 license:mit autotrain_compatible endpoints_compatible text-generation-inference region:us
TAGS
#transformers #pytorch #opt #text-generation #en #arxiv-2303.16755 #license-mit #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 

LM Loss OPT RM
==============


This is a fine tuned OPT 13b model for reward modelling. The finetuning has been done on top of the full SLF5K dataset following the method presented in the paper Training Language Models with Language Feedback at Scale. The main results can be seen in the following table:


Model: OPT LM Loss, # Params: 13B, Validation Accuracy (in %): 73.4 +/- 1.9
Model: OPT LM Loss, # Params: 1.3B, Validation Accuracy (in %): 69.6 +/- 2.0
Model: OPT RM Loss, # Params: 13B, Validation Accuracy (in %): 71.8 +/- 2.0


If using this model, please cite the following paper:



# Params: 13B, Validation Accuracy (in %): 73.4 +/- 1.9
Model: OPT LM Loss, # Params: 1.3B, Validation Accuracy (in %): 69.6 +/- 2.0
Model: OPT RM Loss, # Params: 13B, Validation Accuracy (in %): 71.8 +/- 2.0


If using this model, please cite the following paper:
TAGS
#transformers #pytorch #opt #text-generation #en #arxiv-2303.16755 #license-mit #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 
 # Params: 13B, Validation Accuracy (in %): 73.4 +/- 1.9
Model: OPT LM Loss, # Params: 1.3B, Validation Accuracy (in %): 69.6 +/- 2.0
Model: OPT RM Loss, # Params: 13B, Validation Accuracy (in %): 71.8 +/- 2.0


If using this model, please cite the following paper:
TAGS
#transformers #pytorch #opt #text-generation #en #arxiv-2303.16755 #license-mit #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 
# Params: 13B, Validation Accuracy (in %): 73.4 +/- 1.9
Model: OPT LM Loss, # Params: 1.3B, Validation Accuracy (in %): 69.6 +/- 2.0
Model: OPT RM Loss, # Params: 13B, Validation Accuracy (in %): 71.8 +/- 2.0


If using this model, please cite the following paper: