
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# deciml-7B-rlhf

This model is a fine-tuned version of [Deci/DeciLM-7B](https://huggingface.co/Deci/DeciLM-7B) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.9043
- Rewards/chosen: -0.5041
- Rewards/rejected: -0.2949
- Rewards/accuracies: 0.375
- Rewards/margins: -0.2092
- Logps/rejected: -144.4697
- Logps/chosen: -168.8579
- Logits/rejected: -3.1080
- Logits/chosen: -3.2359

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0002
- train_batch_size: 1
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 2
- training_steps: 50
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
|:-------------:|:-----:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:|:--------------:|:------------:|:---------------:|:-------------:|
| 0.692         | 0.0   | 10   | 0.7242          | -0.1287        | -0.0862          | 0.4375             | -0.0425         | -142.3823      | -165.1031    | -3.0799         | -3.2096       |
| 0.6635        | 0.0   | 20   | 0.7781          | -0.2928        | -0.2069          | 0.4375             | -0.0859         | -143.5897      | -166.7443    | -3.0979         | -3.2283       |
| 0.7188        | 0.0   | 30   | 0.8506          | -0.4155        | -0.2595          | 0.375              | -0.1559         | -144.1155      | -167.9709    | -3.1070         | -3.2367       |
| 0.9092        | 0.0   | 40   | 0.9032          | -0.5122        | -0.2980          | 0.375              | -0.2142         | -144.5009      | -168.9387    | -3.1100         | -3.2382       |
| 0.7197        | 0.0   | 50   | 0.9043          | -0.5041        | -0.2949          | 0.375              | -0.2092         | -144.4697      | -168.8579    | -3.1080         | -3.2359       |


### Framework versions

- Transformers 4.35.2
- Pytorch 2.0.1+cu117
- Datasets 2.15.0
- Tokenizers 0.15.0

{"license": "apache-2.0", "tags": ["generated_from_trainer"], "base_model": "Deci/DeciLM-7B", "model-index": [{"name": "deciml-7B-rlhf", "results": []}]}
tensorboard safetensors generated_from_trainer base_model:Deci/DeciLM-7B license:apache-2.0 region:us