
traversaal-2.5-Mistral-7B is trained via Direct Preference Optimization(DPO) from teknium/OpenHermes-2.5-Mistral-7B as its base model, with several optimizations in hyperparameters.
teknium/OpenHermes-2.5-Mistral-7B is trained via Supervised Fine-Tuning (SFT) using LoRA, with the QWEN-72B model as its base-model.
Note that we did not exploit any form of weight merge.
For leaderboard submission, the trained weight is realigned for compatibility with Mistral-7b



{"license": "apache-2.0", "datasets": ["Intel/orca_dpo_pairs"]}
transformers safetensors mistral text-generation conversational dataset:Intel/orca_dpo_pairs license:apache-2.0 autotrain_compatible endpoints_compatible text-generation-inference region:us