# rpbase

This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).

## Merge Details
### Merge Method

This model was merged using the [DARE](https://arxiv.org/abs/2311.03099) [TIES](https://arxiv.org/abs/2306.01708) merge method using [TheBloke/Llama-2-13B-fp16](https://huggingface.co/TheBloke/Llama-2-13B-fp16) as a base.

### Models Merged

The following models were included in the merge:
* [PygmalionAI/mythalion-13b](https://huggingface.co/PygmalionAI/mythalion-13b)
* [jondurbin/airoboros-l2-13b-2.2.1](https://huggingface.co/jondurbin/airoboros-l2-13b-2.2.1)
* [microsoft/Orca-2-13b](https://huggingface.co/microsoft/Orca-2-13b)
* [ChaiML/phase2_winner_13b2](https://huggingface.co/ChaiML/phase2_winner_13b2)
* [NousResearch/Nous-Hermes-Llama2-13b](https://huggingface.co/NousResearch/Nous-Hermes-Llama2-13b)
* [Norquinal/OpenCAI-13B](https://huggingface.co/Norquinal/OpenCAI-13B)

### Configuration

The following YAML configuration was used to produce this model:

```yaml
models:
- model: Norquinal/OpenCAI-13B
  parameters:
    density: 0.5
    weight: 0.2
- model: PygmalionAI/mythalion-13b
  parameters:
    density: 0.5
    weight: 0.3
- model: microsoft/Orca-2-13b
  parameters:
    density: 0.5
    weight: 0.1
- model: ChaiML/phase2_winner_13b2
  parameters:
    density: 0.5
    weight: 0.4
- model: jondurbin/airoboros-l2-13b-2.2.1
  parameters:
    density: 0.5
    weight: 0.4
- model: NousResearch/Nous-Hermes-Llama2-13b
  parameters:
    density: 0.5
    weight: 0.4
base_model: TheBloke/Llama-2-13B-fp16
merge_method: dare_ties
parameters:
  normalize: 1.0

```

{"tags": ["mergekit", "merge"], "base_model": ["PygmalionAI/mythalion-13b", "TheBloke/Llama-2-13B-fp16", "jondurbin/airoboros-l2-13b-2.2.1", "microsoft/Orca-2-13b", "ChaiML/phase2_winner_13b2", "NousResearch/Nous-Hermes-Llama2-13b", "Norquinal/OpenCAI-13B"]}
transformers safetensors llama text-generation mergekit merge arxiv:2311.03099 arxiv:2306.01708 base_model:PygmalionAI/mythalion-13b base_model:TheBloke/Llama-2-13B-fp16 base_model:jondurbin/airoboros-l2-13b-2.2.1 base_model:microsoft/Orca-2-13b base_model:ChaiML/phase2_winner_13b2 base_model:NousResearch/Nous-Hermes-Llama2-13b base_model:Norquinal/OpenCAI-13B autotrain_compatible endpoints_compatible text-generation-inference region:us