Update README.md
Browse files
README.md
CHANGED
|
@@ -6,7 +6,6 @@ tags:
|
|
| 6 |
- pytorch
|
| 7 |
- lm-head
|
| 8 |
- zh
|
| 9 |
-
datasets:
|
| 10 |
metrics:
|
| 11 |
widget:
|
| 12 |
- text: "小咕噜对靳司寒完全是个自来熟，小家伙爬进他怀里小手搂着他的脖子，奶声奶气的要求：“靳蜀黎,你给咕噜讲故事好不好？”讲故事？童话故事吗？“我不会。”小家伙明显不信。嘟着小嘴大眼汪汪的盯着他，“哼。”小家伙轻轻哼了一声,靳司寒默了半晌，<extra_id_1>"
|
|
@@ -49,6 +48,7 @@ We collect 120G novels as the pretraining data for LongLM.
|
|
| 49 |
```python\
|
| 50 |
from transformers import T5Tokenizer, T5ForConditionalGeneration
|
| 51 |
tokenizer = T5Tokenizer.from_pretrained('LongLM-large')
|
|
|
|
| 52 |
model = T5ForConditionalGeneration.from_pretrained('LongLM-large')
|
| 53 |
```
|
| 54 |
|
|
|
|
| 6 |
- pytorch
|
| 7 |
- lm-head
|
| 8 |
- zh
|
|
|
|
| 9 |
metrics:
|
| 10 |
widget:
|
| 11 |
- text: "小咕噜对靳司寒完全是个自来熟，小家伙爬进他怀里小手搂着他的脖子，奶声奶气的要求：“靳蜀黎,你给咕噜讲故事好不好？”讲故事？童话故事吗？“我不会。”小家伙明显不信。嘟着小嘴大眼汪汪的盯着他，“哼。”小家伙轻轻哼了一声,靳司寒默了半晌，<extra_id_1>"
|
|
|
|
| 48 |
```python\
|
| 49 |
from transformers import T5Tokenizer, T5ForConditionalGeneration
|
| 50 |
tokenizer = T5Tokenizer.from_pretrained('LongLM-large')
|
| 51 |
+
tokenizer.add_special_tokens({"additional_special_tokens": ["<extra_id_%d>"%d for d in range(100)]})
|
| 52 |
model = T5ForConditionalGeneration.from_pretrained('LongLM-large')
|
| 53 |
```
|
| 54 |
|