Training loop for LoRA
Hey all, I wanted to ask if there is any way to apply LoRA on model and then train model in training loop withoutSFTTrainerI am Using the following way, is it fine?model_name = "meta-llama/Meta-Llama-3.1-8B-Instruct"  # or any other model
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    # load_in_8bit=True,
    # quantization_config=quant_config,
    device_map="auto",
    max_memory={0: "4GiB", 1: "50GiB", "cpu": "0.0GiB"},
    offload_state_dict=True,
    low_cpu_mem_usage=True
)

...

model.config.use_cache = False
model.config.pretraining_tp = 1

model.gradient_checkpointing_enable()
model = prepare_model_for_kbit_training(model)

peft_params = LoraConfig(
    r=32,
    lora_alpha=64,
    target_modules=[
        "q_proj",
        "v_proj",
    ],
    lora_dropout=0.05,
    bias="none",
    task_type="CAUSAL_LM"
)

model = get_peft_model(model, peft_params)
model.print_trainable_parameters()
        
optimizer = Adam(model.parameters(), lr=1e-5)

for epoch_idx in [1]:
    for batch_idx, batch in tqdm(enumerate(train_data), total=len(train_data)):
        
        optimizer.zero_grad()
        outputs = model(**inputs)
        ...And then finally usingmerge_and_unload()to merge adapter back to the model
2024-09-17T14:33:24Z
Yes, PEFT is not dependent on the usage ofTrainerorSFTTrainer. Here is just one example of using LoRA with a custom training loop:github.comhuggingface/peft/blob/main/examples/sequence_classification/LoRA.ipynb{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "a9935ae2",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "===================================BUG REPORT===================================\n",
      "Welcome to bitsandbytes. For bug reports, please submit your error trace to: https://github.com/TimDettmers/bitsandbytes/issues\n",
      "For effortless bug reporting copy-paste your error into this form: https://docs.google.com/forms/d/e/1FAIpQLScPB8emS3Thkp66nvqwmjTEgxp8Y9ufuWTzFyr9kJ5AoI47dQ/viewform?usp=sf_link\n",
      "================================================================================\n",
      "CUDA SETUP: CUDA runtime path found: /home/sourab/miniconda3/envs/ml/lib/libcudart.so\n",
      "CUDA SETUP: Highest compute capability among GPUs detected: 7.5\n",
      "CUDA SETUP: Detected CUDA version 117\n",This file has been truncated.show originalDepending on what you would like to do, custom training loops can get a bit complicated (think: mixed precision, gradient accumulation, etc.) but it’s absolutely possible.
2024-09-17T16:21:19Z
Thanks mate. I hope they mention this somewhere the left or Lora page
2024-09-18T04:21:35Z
This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.