File size: 3,035 Bytes
d30b187 1dad186 d30b187 1dad186 d30b187 26e51d8 1969c1e 1dad186 d30b187 1a6e652 d30b187 d571031 d30b187 1dad186 653b2ac 1dad186 26e51d8 1dad186 26e51d8 1dad186 064f784 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 |
---
base_model:
- cognitivecomputations/dolphin-2.9.1-llama-3-70b
library_name: transformers
tags:
- mergekit
- merge
language:
- en
datasets:
- cognitivecomputations/Dolphin-2.9
- teknium/OpenHermes-2.5
- m-a-p/CodeFeedback-Filtered-Instruction
- cognitivecomputations/dolphin-coder
- cognitivecomputations/samantha-data
- HuggingFaceH4/ultrachat_200k
- microsoft/orca-math-word-problems-200k
- abacusai/SystemChat-1.1
- Locutusque/function-calling-chatml
- internlm/Agent-FLAN
---
# merge
This is a merge of [cognitivecomputations/dolphin-2.9.1-llama-3-70b](https://huggingface.co/cognitivecomputations/dolphin-2.9.1-llama-3-70b) into itsself created using [mergekit](https://github.com/cg123/mergekit).
Using the same technique as Llama-3-120b.
## Merge Details
### Merge Method
This model was merged using the passthrough merge method.
### Models Merged
The following models were included in the merge:
* [cognitivecomputations/dolphin-2.9.1-llama-3-70b](https://huggingface.co/cognitivecomputations/dolphin-2.9.1-llama-3-70b)
* [cognitivecomputations/dolphin-2.9.1-llama-3-70b](https://huggingface.co/cognitivecomputations/dolphin-2.9.1-llama-3-70b)
### Configuration
The following YAML configuration was used to produce this model:
```yaml
slices:
- sources:
- layer_range: [0, 20]
model: cognitivecomputations/dolphin-2.9.1-llama-3-70b
- sources:
- layer_range: [10, 30]
model: cognitivecomputations/dolphin-2.9.1-llama-3-70b
- sources:
- layer_range: [20, 40]
model: cognitivecomputations/dolphin-2.9.1-llama-3-70b
- sources:
- layer_range: [30, 50]
model: cognitivecomputations/dolphin-2.9.1-llama-3-70b
- sources:
- layer_range: [40, 60]
model: cognitivecomputations/dolphin-2.9.1-llama-3-70b
- sources:
- layer_range: [50, 70]
model: cognitivecomputations/dolphin-2.9.1-llama-3-70b
- sources:
- layer_range: [60, 80]
model: cognitivecomputations/dolphin-2.9.1-llama-3-70b
merge_method: passthrough
dtype: float16
```
This model uses ChatML prompt template format.
example:
```
<|im_start|>system
You are Dolphin, a helpful AI assistant.<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant
```
## 💻 Usage
```python
!pip install -qU transformers accelerate
from transformers import AutoTokenizer
import transformers
import torch
model = "DarqueDante/LLama-3-Dolphin-2.9.1-120b"
messages = [{"role": "user", "content": "Who is Andrej Karpathy?"}]
tokenizer = AutoTokenizer.from_pretrained(model)
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
pipeline = transformers.pipeline(
"text-generation",
model=model,
torch_dtype=torch.float16,
device_map="auto",
)
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])
```
## 💻 Testing
I have not ran any benchmarking tests on the model yet, though that is my goal for this holiday weekend.
|