Update README.md
Browse files
README.md
CHANGED
@@ -6,22 +6,24 @@ pipeline_tag: text-generation
|
|
6 |
base_model:
|
7 |
- allenai/OLMo-2-13B-1124
|
8 |
library_name: transformers
|
|
|
|
|
9 |
---
|
10 |
|
11 |
<img alt="OLMo Logo" src="https://huggingface.co/datasets/allenai/blog-images/resolve/main/olmo2/olmo.png" width="242px">
|
12 |
|
13 |
# OLMo-2-1124-13B-SFT
|
14 |
|
15 |
-
OLMo
|
16 |
Tülu 3 is designed for state-of-the-art performance on a diversity of tasks in addition to chat, such as MATH, GSM8K, and IFEval.
|
17 |
-
Check out
|
18 |
|
19 |
OLMo is a series of **O**pen **L**anguage **Mo**dels designed to enable the science of language models.
|
20 |
These models are trained on the Dolma dataset. We are releasing all code, checkpoints, logs (coming soon), and associated training details.
|
21 |
The core models released in this batch include the following:
|
22 |
|
23 |
|
24 |
-
| **Stage** | **OLMo
|
25 |
|----------------------|----------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------|
|
26 |
| **Base Model** | [allenai/OLMo2-7B-1124](https://huggingface.co/allenai/OLMo2-7B-1124) | [allenai/OLMo-2-13B-1124](https://huggingface.co/allenai/OLMo-2-13B-1124) |
|
27 |
| **SFT** | [allenai/OLMo-2-1124-7B-SFT](https://huggingface.co/allenai/OLMo-2-1124-7B-SFT) | [allenai/OLMo-2-1124-13B-SFT](https://huggingface.co/allenai/OLMo-2-1124-13B-SFT) |
|
@@ -45,7 +47,7 @@ The core models released in this batch include the following:
|
|
45 |
- Core repo (training, inference, fine-tuning etc.): https://github.com/allenai/OLMo
|
46 |
- Evaluation code: https://github.com/allenai/olmes
|
47 |
- Further fine-tuning code: https://github.com/allenai/open-instruct
|
48 |
-
- **Paper:** Coming soon!
|
49 |
- **Demo:** https://playground.allenai.org/
|
50 |
|
51 |
## Using the model
|
@@ -84,7 +86,7 @@ The model has not been trained with a specific system prompt in mind.
|
|
84 |
|
85 |
### Bias, Risks, and Limitations
|
86 |
|
87 |
-
The OLMo
|
88 |
See the Falcon 180B model card for an example of this.
|
89 |
|
90 |
|
@@ -105,13 +107,13 @@ SFT:
|
|
105 |
|
106 |
## License and use
|
107 |
|
108 |
-
OLMo
|
109 |
-
OLMo
|
110 |
For more information, please see our [Responsible Use Guidelines](https://allenai.org/responsible-use).
|
111 |
|
112 |
## Citation
|
113 |
|
114 |
-
If OLMo
|
115 |
```
|
116 |
TODO
|
117 |
```
|
|
|
6 |
base_model:
|
7 |
- allenai/OLMo-2-13B-1124
|
8 |
library_name: transformers
|
9 |
+
datasets:
|
10 |
+
- allenai/tulu-3-sft-olmo-2-mixture
|
11 |
---
|
12 |
|
13 |
<img alt="OLMo Logo" src="https://huggingface.co/datasets/allenai/blog-images/resolve/main/olmo2/olmo.png" width="242px">
|
14 |
|
15 |
# OLMo-2-1124-13B-SFT
|
16 |
|
17 |
+
OLMo 2 13B SFT November 2024 is post-trained variant of the [OLMo-2 13B November 2024](https://huggingface.co/allenai/OLMo2-13B-1124) model, which has undergone supervised finetuning on the [Tülu 3 dataset](https://huggingface.co/datasets/allenai/tulu-3-sft-olmo-2-mixture).
|
18 |
Tülu 3 is designed for state-of-the-art performance on a diversity of tasks in addition to chat, such as MATH, GSM8K, and IFEval.
|
19 |
+
Check out the OLMo 2 paper (forthcoming) or [Tülu 3 paper](https://arxiv.org/abs/2411.15124) for more details!
|
20 |
|
21 |
OLMo is a series of **O**pen **L**anguage **Mo**dels designed to enable the science of language models.
|
22 |
These models are trained on the Dolma dataset. We are releasing all code, checkpoints, logs (coming soon), and associated training details.
|
23 |
The core models released in this batch include the following:
|
24 |
|
25 |
|
26 |
+
| **Stage** | **OLMo 2 7B** | **OLMo 2 7B** |
|
27 |
|----------------------|----------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------|
|
28 |
| **Base Model** | [allenai/OLMo2-7B-1124](https://huggingface.co/allenai/OLMo2-7B-1124) | [allenai/OLMo-2-13B-1124](https://huggingface.co/allenai/OLMo-2-13B-1124) |
|
29 |
| **SFT** | [allenai/OLMo-2-1124-7B-SFT](https://huggingface.co/allenai/OLMo-2-1124-7B-SFT) | [allenai/OLMo-2-1124-13B-SFT](https://huggingface.co/allenai/OLMo-2-1124-13B-SFT) |
|
|
|
47 |
- Core repo (training, inference, fine-tuning etc.): https://github.com/allenai/OLMo
|
48 |
- Evaluation code: https://github.com/allenai/olmes
|
49 |
- Further fine-tuning code: https://github.com/allenai/open-instruct
|
50 |
+
- **Paper:** Coming soon!
|
51 |
- **Demo:** https://playground.allenai.org/
|
52 |
|
53 |
## Using the model
|
|
|
86 |
|
87 |
### Bias, Risks, and Limitations
|
88 |
|
89 |
+
The OLMo 2 models have limited safety training, but are not deployed automatically with in-the-loop filtering of responses like ChatGPT, so the model can produce problematic outputs (especially when prompted to do so).
|
90 |
See the Falcon 180B model card for an example of this.
|
91 |
|
92 |
|
|
|
107 |
|
108 |
## License and use
|
109 |
|
110 |
+
OLMo 2 is licensed under the Apache 2.0 license.
|
111 |
+
OLMo 2 is intended for research and educational use.
|
112 |
For more information, please see our [Responsible Use Guidelines](https://allenai.org/responsible-use).
|
113 |
|
114 |
## Citation
|
115 |
|
116 |
+
If OLMo 2 or any of the related materials were helpful to your work, please cite:
|
117 |
```
|
118 |
TODO
|
119 |
```
|