transformers pytorch opt text-generation generated_from_trainer autotrain_compatible endpoints_compatible text-generation-inference region:us
---
tags:
- generated_from_trainer
model-index:
- name: rel-cl-transformer-4
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# rel-cl-transformer-4

This model is a fine-tuned version of [](https://huggingface.co/) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 3.8664

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 32
- eval_batch_size: 32
- seed: 4
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- training_steps: 3052726

### Training results

| Training Loss | Epoch | Step    | Validation Loss |
|:-------------:|:-----:|:-------:|:---------------:|
| 4.2388        | 0.03  | 76319   | 4.2044          |
| 4.0326        | 0.03  | 152638  | 4.0318          |
| 3.9236        | 0.03  | 228957  | 3.9580          |
| 3.8544        | 0.03  | 305276  | 3.9166          |
| 3.8024        | 0.03  | 381595  | 3.8908          |
| 3.7553        | 0.03  | 457914  | 3.8751          |
| 3.7234        | 0.03  | 534233  | 3.8648          |
| 3.6968        | 1.03  | 610552  | 3.8583          |
| 3.6689        | 0.03  | 686872  | 3.8499          |
| 3.6414        | 1.03  | 763192  | 3.8469          |
| 3.6187        | 0.03  | 839512  | 3.8467          |
| 3.5985        | 1.03  | 915832  | 3.8456          |
| 3.5764        | 0.03  | 992152  | 3.8459          |
| 3.5579        | 1.03  | 1068472 | 3.8476          |
| 3.5454        | 0.03  | 1144792 | 3.8477          |
| 3.5313        | 0.03  | 1221112 | 3.8497          |
| 3.521         | 1.03  | 1297432 | 3.8510          |
| 3.5044        | 0.03  | 1373752 | 3.8522          |
| 3.4925        | 1.03  | 1450072 | 3.8531          |
| 3.4831        | 0.03  | 1526392 | 3.8551          |
| 3.4716        | 1.03  | 1602712 | 3.8569          |
| 3.4625        | 0.03  | 1679032 | 3.8572          |
| 3.456         | 1.03  | 1755352 | 3.8592          |
| 3.4435        | 0.03  | 1831672 | 3.8614          |
| 3.43          | 1.03  | 1907992 | 3.8618          |
| 3.4152        | 0.03  | 1984312 | 3.8625          |
| 3.4052        | 1.03  | 2060632 | 3.8641          |
| 3.3947        | 0.03  | 2136952 | 3.8649          |
| 3.3836        | 1.03  | 2213272 | 3.8653          |
| 3.3684        | 0.03  | 2289592 | 3.8668          |
| 3.3616        | 0.03  | 2365912 | 3.8675          |
| 3.3559        | 1.03  | 2442232 | 3.8683          |
| 3.3476        | 0.03  | 2518552 | 3.8687          |
| 3.3375        | 1.03  | 2594872 | 3.8693          |
| 3.3274        | 0.03  | 2671192 | 3.8691          |
| 3.321         | 1.03  | 2747512 | 3.8689          |
| 3.3125        | 0.03  | 2823832 | 3.8684          |
| 3.3092        | 1.03  | 2900152 | 3.8682          |
| 3.3039        | 0.03  | 2976472 | 3.8670          |
| 3.2951        | 1.02  | 3052726 | 3.8664          |


### Framework versions

- Transformers 4.33.3
- Pytorch 2.0.1
- Datasets 2.12.0
- Tokenizers 0.13.3
