transformers safetensors llama text-generation conversational autotrain_compatible endpoints_compatible text-generation-inference region:us
---
library_name: transformers
tags: []
---
```
training_arguments = TrainingArguments(
    output_dir= "./results",
    num_train_epochs= 6,
    per_device_train_batch_size= 64,
    gradient_accumulation_steps= 16,
    optim = "paged_adamw_8bit",
    save_steps= 3,
    logging_steps= 1,
    learning_rate= 4e-4,
    weight_decay= 0.001,
    fp16= False,
    bf16= False,
    max_grad_norm= 0.3,
    max_steps= -1,
    warmup_ratio= 0.3,
    group_by_length= True,
    lr_scheduler_type= "linear",
    report_to="wandb",
)
```

```
Run history:

train/epoch	▁▁▁▂▂▂▂▃▃▃▃▄▄▄▄▅▅▅▅▆▆▆▆▇▇▇▇████
train/global_step	▁▁▁▂▂▂▂▃▃▃▃▄▄▄▄▅▅▅▅▆▆▆▆▇▇▇▇████
train/learning_rate	▂▃▃▄▅▆▆▇██▇▇▇▆▆▆▅▅▅▄▄▄▃▃▃▂▂▂▁▁
train/loss	██▇▇▆▅▅▄▃▃▃▃▃▃▂▂▂▂▂▂▂▂▁▁▁▁▁▁▁▁
train/total_flos	▁
train/train_loss	▁
train/train_runtime	▁
train/train_samples_per_second	▁
train/train_steps_per_second	▁

Run summary:

train/epoch	5.16
train/global_step	30
train/learning_rate	0.0
train/loss	1.1752
train/total_flos	7297433452388352.0
train/train_loss	1.49426
train/train_runtime	2373.0384
train/train_samples_per_second	14.971
train/train_steps_per_second	0.013
```