metadata

base_model: meta-llama/meta-llama-3.1-8b-instruct
language:
  - am
  - ha
  - ig
  - rw
  - st
  - sn
  - so
  - sw
  - xh
  - yo
  - zu
  - en
  - fr
license: apache-2.0
tags:
  - text-generation-inference
  - transformers
  - unsloth
  - llama
  - trl
datasets:
  - masakhane/african-ultrachat

Llama-3.1-8B Instruct African Ultrachat

Developed by: vutuka
License: apache-2.0
Finetuned from model : meta-llama/meta-llama-3.1-8b-instruct
Max Content Length : 8192
Max Steps : 800
Training Time : 02h-22min-08s
Setup :
- 1 x RTX A6000
- 16 vCPU
- 58 GB RAM
- 150 GB Storage

Introducing Llama 3.1-8B Instruct Fine-Tuned on the Masakhane African UltraChat Dataset

We are excited to announce the fine-tuned version of the Llama 3.1-8B Instruct model, which has been trained on the Masakhane African UltraChat dataset. This fine-tuning leverages the robust architecture of the Llama 3.1 model, designed for high-performance multilingual tasks and long context processing, to enhance its capabilities in understanding and generating responses in African languages.

Model Overview

Llama 3.1-8B Instruct is part of the Llama 3 family, developed by Meta. It features an optimized transformer architecture and supports multiple languages, including English, German, French, Italian, Portuguese, Hindi, Spanish, and Thai. This model variant is particularly suited for instruction-tuned tasks, making it ideal for dialogue and assistant-like applications.

Training and Fine-Tuning

The model was fine-tuned using the Masakhane African UltraChat dataset, which is a diverse and extensive collection of conversational data aimed at promoting and enhancing NLP capabilities for African languages. The fine-tuning process involved supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF), ensuring the model aligns well with human preferences for helpfulness and safety.

trainer = SFTTrainer(
    model = model,
    tokenizer = tokenizer,
    train_dataset = shuffled_dataset,
    dataset_text_field = "text",
    max_seq_length = max_seq_length,
    dataset_num_proc = 2,
    packing = False, # Can make training 5x faster for short sequences.
    args = TrainingArguments(
        per_device_train_batch_size = 2,
        gradient_accumulation_steps = 4,
        warmup_steps = 5,
        max_steps = 800,
        do_eval=True,
        learning_rate = 3e-4,
        log_level="debug",
        #fp16 = not is_bfloat16_supported(),
        bf16 = True,
        logging_steps = 10,
        optim = "adamw_8bit",
        weight_decay = 0.01,
        lr_scheduler_type = "linear",
        seed = 3407,
        output_dir = "outputs",
        report_to='wandb',
        warmup_ratio=0.3,
    ),
)

Performance and Capabilities

The fine-tuned Llama 3.1-8B model demonstrates improved performance in understanding and generating text in African languages, providing accurate and contextually appropriate responses. It is designed to handle various conversational tasks, from casual dialogue to more complex inquiries, making it a valuable tool for applications targeting African language users.

Key Features

Multilingual Support: Enhanced capabilities in multiple languages, including African languages.
Long Context Handling: Supports up to 128k tokens, making it suitable for long-form conversations.
Instruction-Tuned: Optimized for generating accurate and helpful responses based on user instructions.
High Performance: Utilizes advanced techniques like Grouped-Query Attention (GQA) for improved scalability and efficiency.

Tokenizer & Chat Format

from unsloth.chat_templates import get_chat_template

tokenizer = get_chat_template(
    tokenizer,
    chat_template = "llama-3", # Supports zephyr, chatml, mistral, llama, alpaca, vicuna, vicuna_old, unsloth
    mapping={
        "role": "role",
        "content": "content",
        "user": "",
        "assistant": "",
    }
)

def formatting_prompts_func(examples):
    convos = examples["messages"]
    texts = [tokenizer.apply_chat_template(convo, tokenize = False, add_generation_prompt = False) for convo in convos]
    return { "text" : texts, }
pass

Inference with Unsloth

def chat_llama3_african_ultrachat(message: str, context: str):
    FastLanguageModel.for_inference(model) # Enable native 2x faster inference
    
    messages = [
        {"role": "user", "content": context},
        {"role": "user", "content": message},
    ]
    inputs = tokenizer.apply_chat_template(
        messages,
        tokenize = True,
        add_generation_prompt = True, # Must add for generation
        return_tensors = "pt",
    ).to("cuda")
    
    from transformers import TextStreamer
    text_streamer = TextStreamer(tokenizer)
    _ = model.generate(input_ids = inputs, streamer = text_streamer, max_new_tokens = 1024, use_cache = True)

chat_llama3_african_ultrachat(
    message="Habari !",
    context="Wewe ni wakala wa mtandaoni anayesaidia ambaye hujibu maswali kwa upole na heshima."
)

<|begin_of_text|><|start_header_id|>user<|end_header_id|>

Wewe ni wakala wa mtandaoni anayesaidia ambaye hujibu maswali kwa upole na heshima.<|eot_id|><|start_header_id|>user<|end_header_id|>

Habari!<|eot_id|><|start_header_id|>assistant<|end_header_id|>

Habari yako? Je, unatafuta ushauri au maelezo kuhusu jambo maalum? Ni furaha yangu kusaidia.<|eot_id|>

This llama model was trained 2x faster with Unsloth and Huggingface's TRL library.