


# Instruction tune of Yi-34b-200k with Airoboros-3.1 (fp16)


## Overview

This is [larryvrh/Yi-34B-200K-Llamafied](https://huggingface.co/larryvrh/Yi-34B-200K-Llamafied), with instruction tuning performed with Jon Durbin's [jondurbin/airoboros-3.1](https://huggingface.co/datasets/jondurbin/airoboros-3.1) dataset. That base model is [01-ai/Yi-34B-200k](https://huggingface.co/01-ai/Yi-34B-200k), but using llama2 model definitions and tokenizer to remove any remote code requirements.

**This is a (merged) QLoRA fine-tune (rank 64)**. 

The finetune was performed with 1x RTX 6000 Ada (~80 hours to this checkpoint). Prompts were truncated to 4096 tokens (for speed and VRAM headroom).

I have done very little testing with this model, so feedback on real world performance is appreciated!

## How to Use

Use as you would any other Hugging Face fp16 llama-2 model.

## Prompting:

Model was trained with llama-2 chat prompt format. See [jondurbin/airoboros-l2-13b-3.1.1](https://huggingface.co/jondurbin/airoboros-l2-13b-3.1.1) model card for details.


{"license": "other", "datasets": ["jondurbin/airoboros-3.1"], "license_name": "yi-license", "license_link": "LICENSE"}
transformers safetensors llama text-generation dataset:jondurbin/airoboros-3.1 license:other autotrain_compatible endpoints_compatible text-generation-inference region:us