transformers pytorch safetensors llama text-generation pretrained en license:apache-2.0 autotrain_compatible endpoints_compatible text-generation-inference region:us
---
license: apache-2.0
pipeline_tag: text-generation
language:
  - en
tags:
- pretrained
inference:
  parameters:
    temperature: 0.7
---

This is a simple port of [mistralai/Mistral-7B-v0.1](mistralai/Mistral-7B-v0.1) with modified configuration to fix the broken rope scaling from SWA present in v0.1 and fine-tunes of v0.1. 

The fix leverages the same strategy employed by [upstage/SOLAR-10.7B-v1.0](https://huggingface.co/upstage/SOLAR-10.7B-v1.0) where the model config is adjusted to imitate a llama model with 4k context. 