
Sorry everyone this got sort of popular but it doesnt generate understandable text - I think there's a way to make this generate good results w/ relatively little compute I'll experiment a bit later

# LMLab Mistral 1B Untrained

This is an untrained base model modified from Mistral-7B-Instruct. It has 1.13 billion parameters.

## Untrained

This model is untrained. **This means it will not generate understandable text.**

## Model Details

### Model Description

- **Developed by:** LMLab
- **License:** Apache 2.0
- **Parameters:** 1.13 billion (1,134,596,096)
- **Modified from model:** [`mistralai/Mistral-7B-v0.1`](https://huggingface.co/mistralai/Mistral-7B-v0.1)

### Model Architecture

LMLab Mistral 1B is a transformer model, with the following architecture choices:

* Grouped-Query Attention
* Sliding-Window Attention
* Byte-fallback BPE tokenizer

## Usage

Use `MistralForCausalLM`.

```python
from transformers import MistralForCausalLM, AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained('lmlab/lmlab-mistral-1b-untrained')
model = MistralForCausalLM.from_pretrained('lmlab/lmlab-mistral-1b-untrained')
text = "Once upon a time"
encoded_input = tokenizer(text, return_tensors='pt')
output = model.generate(**encoded_input)
print(tokenizer.decode(output[0]))
```

## Notice

This model does not have any moderation systems.
{"language": ["en"], "license": "apache-2.0", "pipeline_tag": "text-generation"}
transformers safetensors mistral text-generation en license:apache-2.0 autotrain_compatible endpoints_compatible text-generation-inference region:us