amanrangapur
commited on
Update README.md
Browse files
README.md
CHANGED
@@ -19,16 +19,16 @@ The core models released in this batch include the following:
|
|
19 |
|
20 |
| Size | Training Tokens | Layers | Hidden Size | Attention Heads | Context Length |
|
21 |
|------|--------|---------|-------------|-----------------|----------------|
|
22 |
-
| [OLMo2-7B
|
23 |
-
| [OLMo2- 13B
|
24 |
|
25 |
## Inference
|
26 |
|
27 |
You can use OLMo with the standard HuggingFace transformers library:
|
28 |
```python
|
29 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
30 |
-
olmo = AutoModelForCausalLM.from_pretrained("allenai/
|
31 |
-
tokenizer = AutoTokenizer.from_pretrained("allenai/
|
32 |
message = ["Language modeling is "]
|
33 |
inputs = tokenizer(message, return_tensors='pt', return_token_type_ids=False)
|
34 |
# optional verifying cuda
|
@@ -41,7 +41,7 @@ print(tokenizer.batch_decode(response, skip_special_tokens=True)[0])
|
|
41 |
|
42 |
For faster performance, you can quantize the model using the following method:
|
43 |
```python
|
44 |
-
AutoModelForCausalLM.from_pretrained("allenai/
|
45 |
torch_dtype=torch.float16,
|
46 |
load_in_8bit=True) # Requires bitsandbytes package
|
47 |
```
|
@@ -55,13 +55,13 @@ The naming convention is `stepXXX-tokensYYYB`.
|
|
55 |
|
56 |
To load a specific model revision with HuggingFace, simply add the argument `revision`:
|
57 |
```bash
|
58 |
-
olmo = AutoModelForCausalLM.from_pretrained("allenai/
|
59 |
```
|
60 |
|
61 |
Or, you can access all the revisions for the models via the following code snippet:
|
62 |
```python
|
63 |
from huggingface_hub import list_repo_refs
|
64 |
-
out = list_repo_refs("allenai/
|
65 |
branches = [b.name for b in out.branches]
|
66 |
```
|
67 |
|
|
|
19 |
|
20 |
| Size | Training Tokens | Layers | Hidden Size | Attention Heads | Context Length |
|
21 |
|------|--------|---------|-------------|-----------------|----------------|
|
22 |
+
| [OLMo2-7B](https://huggingface.co/allenai/OLMo-1124-7B) | 4 Trillion | 32 | 4096 | 32 | 4096 |
|
23 |
+
| [OLMo2- 13B](https://huggingface.co/allenai/OLMo2-1124-13B) | 5 Trillion | 40 | 5120 | 42 | 4096 |
|
24 |
|
25 |
## Inference
|
26 |
|
27 |
You can use OLMo with the standard HuggingFace transformers library:
|
28 |
```python
|
29 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
30 |
+
olmo = AutoModelForCausalLM.from_pretrained("allenai/OLMo-2-1124-13B")
|
31 |
+
tokenizer = AutoTokenizer.from_pretrained("allenai/OLMo-2-1124-13B")
|
32 |
message = ["Language modeling is "]
|
33 |
inputs = tokenizer(message, return_tensors='pt', return_token_type_ids=False)
|
34 |
# optional verifying cuda
|
|
|
41 |
|
42 |
For faster performance, you can quantize the model using the following method:
|
43 |
```python
|
44 |
+
AutoModelForCausalLM.from_pretrained("allenai/OLMo-2-1124-13B",
|
45 |
torch_dtype=torch.float16,
|
46 |
load_in_8bit=True) # Requires bitsandbytes package
|
47 |
```
|
|
|
55 |
|
56 |
To load a specific model revision with HuggingFace, simply add the argument `revision`:
|
57 |
```bash
|
58 |
+
olmo = AutoModelForCausalLM.from_pretrained("allenai/OLMo-2-1124-13B", revision="step102500-tokens860B")
|
59 |
```
|
60 |
|
61 |
Or, you can access all the revisions for the models via the following code snippet:
|
62 |
```python
|
63 |
from huggingface_hub import list_repo_refs
|
64 |
+
out = list_repo_refs("allenai/OLMo-2-1124-13B")
|
65 |
branches = [b.name for b in out.branches]
|
66 |
```
|
67 |
|