Text Generation
Transformers
PyTorch
English
olmo2
conversational
Inference Endpoints
hamishivi commited on
Commit
9556d9b
·
verified ·
1 Parent(s): f3251a7

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +10 -8
README.md CHANGED
@@ -6,22 +6,24 @@ pipeline_tag: text-generation
6
  base_model:
7
  - allenai/OLMo-2-13B-1124
8
  library_name: transformers
 
 
9
  ---
10
 
11
  <img alt="OLMo Logo" src="https://huggingface.co/datasets/allenai/blog-images/resolve/main/olmo2/olmo.png" width="242px">
12
 
13
  # OLMo-2-1124-13B-SFT
14
 
15
- OLMo-2 13B SFT November 2024 is finetuned variant of the [OLMo-2 13B November 2024](https://huggingface.co/allenai/OLMo2-13B-1124) model, which has undergone supervised finetuning on the [Tülu 3 dataset](https://huggingface.co/datasets/allenai/tulu-3-sft-mixture).
16
  Tülu 3 is designed for state-of-the-art performance on a diversity of tasks in addition to chat, such as MATH, GSM8K, and IFEval.
17
- Check out [the OLMo-2 paper](https://TODO) or [Tülu 3 paper](https://arxiv.org/abs/2411.15124) for more details!
18
 
19
  OLMo is a series of **O**pen **L**anguage **Mo**dels designed to enable the science of language models.
20
  These models are trained on the Dolma dataset. We are releasing all code, checkpoints, logs (coming soon), and associated training details.
21
  The core models released in this batch include the following:
22
 
23
 
24
- | **Stage** | **OLMo-2 7B** | **OLMo-2 7B** |
25
  |----------------------|----------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------|
26
  | **Base Model** | [allenai/OLMo2-7B-1124](https://huggingface.co/allenai/OLMo2-7B-1124) | [allenai/OLMo-2-13B-1124](https://huggingface.co/allenai/OLMo-2-13B-1124) |
27
  | **SFT** | [allenai/OLMo-2-1124-7B-SFT](https://huggingface.co/allenai/OLMo-2-1124-7B-SFT) | [allenai/OLMo-2-1124-13B-SFT](https://huggingface.co/allenai/OLMo-2-1124-13B-SFT) |
@@ -45,7 +47,7 @@ The core models released in this batch include the following:
45
  - Core repo (training, inference, fine-tuning etc.): https://github.com/allenai/OLMo
46
  - Evaluation code: https://github.com/allenai/olmes
47
  - Further fine-tuning code: https://github.com/allenai/open-instruct
48
- - **Paper:** Coming soon! TODO
49
  - **Demo:** https://playground.allenai.org/
50
 
51
  ## Using the model
@@ -84,7 +86,7 @@ The model has not been trained with a specific system prompt in mind.
84
 
85
  ### Bias, Risks, and Limitations
86
 
87
- The OLMo-2 models have limited safety training, but are not deployed automatically with in-the-loop filtering of responses like ChatGPT, so the model can produce problematic outputs (especially when prompted to do so).
88
  See the Falcon 180B model card for an example of this.
89
 
90
 
@@ -105,13 +107,13 @@ SFT:
105
 
106
  ## License and use
107
 
108
- OLMo-2 is licensed under the Apache 2.0 license.
109
- OLMo-2 is intended for research and educational use.
110
  For more information, please see our [Responsible Use Guidelines](https://allenai.org/responsible-use).
111
 
112
  ## Citation
113
 
114
- If OLMo-2 or any of the related materials were helpful to your work, please cite:
115
  ```
116
  TODO
117
  ```
 
6
  base_model:
7
  - allenai/OLMo-2-13B-1124
8
  library_name: transformers
9
+ datasets:
10
+ - allenai/tulu-3-sft-olmo-2-mixture
11
  ---
12
 
13
  <img alt="OLMo Logo" src="https://huggingface.co/datasets/allenai/blog-images/resolve/main/olmo2/olmo.png" width="242px">
14
 
15
  # OLMo-2-1124-13B-SFT
16
 
17
+ OLMo 2 13B SFT November 2024 is post-trained variant of the [OLMo-2 13B November 2024](https://huggingface.co/allenai/OLMo2-13B-1124) model, which has undergone supervised finetuning on the [Tülu 3 dataset](https://huggingface.co/datasets/allenai/tulu-3-sft-olmo-2-mixture).
18
  Tülu 3 is designed for state-of-the-art performance on a diversity of tasks in addition to chat, such as MATH, GSM8K, and IFEval.
19
+ Check out the OLMo 2 paper (forthcoming) or [Tülu 3 paper](https://arxiv.org/abs/2411.15124) for more details!
20
 
21
  OLMo is a series of **O**pen **L**anguage **Mo**dels designed to enable the science of language models.
22
  These models are trained on the Dolma dataset. We are releasing all code, checkpoints, logs (coming soon), and associated training details.
23
  The core models released in this batch include the following:
24
 
25
 
26
+ | **Stage** | **OLMo 2 7B** | **OLMo 2 7B** |
27
  |----------------------|----------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------|
28
  | **Base Model** | [allenai/OLMo2-7B-1124](https://huggingface.co/allenai/OLMo2-7B-1124) | [allenai/OLMo-2-13B-1124](https://huggingface.co/allenai/OLMo-2-13B-1124) |
29
  | **SFT** | [allenai/OLMo-2-1124-7B-SFT](https://huggingface.co/allenai/OLMo-2-1124-7B-SFT) | [allenai/OLMo-2-1124-13B-SFT](https://huggingface.co/allenai/OLMo-2-1124-13B-SFT) |
 
47
  - Core repo (training, inference, fine-tuning etc.): https://github.com/allenai/OLMo
48
  - Evaluation code: https://github.com/allenai/olmes
49
  - Further fine-tuning code: https://github.com/allenai/open-instruct
50
+ - **Paper:** Coming soon!
51
  - **Demo:** https://playground.allenai.org/
52
 
53
  ## Using the model
 
86
 
87
  ### Bias, Risks, and Limitations
88
 
89
+ The OLMo 2 models have limited safety training, but are not deployed automatically with in-the-loop filtering of responses like ChatGPT, so the model can produce problematic outputs (especially when prompted to do so).
90
  See the Falcon 180B model card for an example of this.
91
 
92
 
 
107
 
108
  ## License and use
109
 
110
+ OLMo 2 is licensed under the Apache 2.0 license.
111
+ OLMo 2 is intended for research and educational use.
112
  For more information, please see our [Responsible Use Guidelines](https://allenai.org/responsible-use).
113
 
114
  ## Citation
115
 
116
+ If OLMo 2 or any of the related materials were helpful to your work, please cite:
117
  ```
118
  TODO
119
  ```