File size: 842 Bytes
d444b74 0383df1 02dc790 d444b74 02dc790 0383df1 02dc790 0383df1 02dc790 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 |
---
library_name: peft
license: apache-2.0
datasets:
- Abirate/english_quotes
language:
- en
pipeline_tag: text-generation
---
## Base model
bigscience/bloomz-560m
## Training procedure
According to edX Databricks llm102 course
### PromptTuningConfig
- task_type=TaskType.CAUSAL_LM,
- prompt_tuning_init=PromptTuningInit.RANDOM,
- num_virtual_tokens=4,
### TrainingArguments
- learning_rate= 3e-2, # Higher learning rate than full fine-tuning
- num_train_epochs=5 # Number of passes to go through the entire fine-tuning dataset
### Framework versions
- PEFT 0.4.0
### Training output
TrainOutput(global_step=35, training_loss=3.386413792201451, metrics={'train_runtime': 617.1546, 'train_samples_per_second': 0.405, 'train_steps_per_second': 0.057, 'total_flos': 58327152033792.0, 'train_loss': 3.386413792201451, 'epoch': 5.0})
|