
# This is the Token customized model with a Context Length of 1M Tokens

This is based on LWMs Text-1M model with added custom special Tokens.

<br>

### New added Special Tokens

```text
"<|functions|>",
"<|gökdeniz|>",
"<|user|>",
"<|josie|>",
"<|assistant|>",
"<|function_call|>",
"<|function_response|>",
"<|image|>",
"<|long_term_memory|>",
"<|home_state|>",
"<|endoftext|>"
```
<br>

### New added Normal Tokens
  
```text
"Gökdeniz Gülmez"
"Gökdeniz"
"Gülmez"
"JOSIE"
"J.O.S.I.E."
"Josie"
"josie"
"Just an Outstandingly Smart and Intelligent Entity"
```
<br>
<br>

## Model details

**Model type:**
LWM-Text-1M is an open-source model trained from LLaMA-2 on a subset of Books3 filtered data. It is an auto-regressive language model, based on the transformer architecture.

**Model date:**
LWM-Text-1M was trained in December 2023.

**Paper or resources for more information:**
https://largeworldmodel.github.io/

## License
Llama 2 is licensed under the LLAMA 2 Community License, 
Copyright (c) Meta Platforms, Inc. All Rights Reserved.

**Where to send questions or comments about the model:**
https://github.com/LargeWorldModel/lwm/issues

## Training dataset
- 800 subset of Books3 documents with 1M plus tokens
{"inference": true, "pipeline_tag": "text-generation"}
transformers safetensors llama text-generation autotrain_compatible endpoints_compatible text-generation-inference region:us