--- base_model: /data/junxiong/sft/zephyr_0_5_sft_open_not_openhermes_progressive_train_largest_dataset/ tags: - alignment-handbook - generated_from_trainer datasets: - HuggingFaceH4/ultrafeedback_binarized model-index: - name: zephyr_0_5_dpo_open_not_openhermes_progressive_train_largest_dataset_ep3 results: [] --- # zephyr_0_5_dpo_open_not_openhermes_progressive_train_largest_dataset_ep3 This model is a fine-tuned version of [/data/junxiong/sft/zephyr_0_5_sft_open_not_openhermes_progressive_train_largest_dataset/](https://huggingface.co//data/junxiong/sft/zephyr_0_5_sft_open_not_openhermes_progressive_train_largest_dataset/) on the HuggingFaceH4/ultrafeedback_binarized dataset. It achieves the following results on the evaluation set: - Loss: 0.7141 - Rewards/chosen: -5.3346 - Rewards/rejected: -8.3118 - Rewards/accuracies: 0.7891 - Rewards/margins: 2.9772 - Logps/rejected: -337.4994 - Logps/chosen: -304.9619 - Logits/rejected: -2.7812 - Logits/chosen: -2.8272 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 5e-07 - train_batch_size: 4 - eval_batch_size: 8 - seed: 42 - distributed_type: multi-GPU - num_devices: 8 - total_train_batch_size: 32 - total_eval_batch_size: 64 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: cosine - lr_scheduler_warmup_ratio: 0.1 - num_epochs: 3 ### Training results | Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen | |:-------------:|:------:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:|:--------------:|:------------:|:---------------:|:-------------:| | 0.1171 | 1.0466 | 2000 | 0.5329 | -1.4521 | -2.9272 | 0.7734 | 1.4750 | -283.6535 | -266.1376 | -2.8897 | -2.9362 | | 0.0086 | 2.0931 | 4000 | 0.7141 | -5.3346 | -8.3118 | 0.7891 | 2.9772 | -337.4994 | -304.9619 | -2.7812 | -2.8272 | ### Framework versions - Transformers 4.41.2 - Pytorch 2.1.0+cu118 - Datasets 2.20.0 - Tokenizers 0.19.1