


# Mistral-7b-DZ_Startups

## Model Information : 
+ **Model Name**: Mistral-7b-DZ_Startups
+ **Model Architecture**: Mistral-7B-Instruct-v0.1
+ **Model Size**: 7.3 billion parameters
+ **Fine-tuned on**: Algerian Startups Dataset V1  by TuningAI Team

## Limits : 
+ **Sequence Length**: Due to the use of Sliding Window Attention (SWA), the maximum sequence length may be limited. Ensure that your input data conforms to the model's sequence length restrictions.
+ **Language Support**: The model's primary strength lies in English tasks, but it has been fine-tuned on Algerian Startups data, so its performance in English and code-related tasks may be more reliable.
+ **Fine-tuning Data**: The model's performance is tied to the quality and diversity of the fine-tuning data. Results may vary on tasks unrelated to Algerian startups.



## Ethics considerations : 
+ **Bias Mitigation**: Efforts have been made to reduce bias during fine-tuning, but it's essential to be aware of potential biases in the data and to evaluate model outputs critically.
+ **Fair Use**: Please use this model responsibly and avoid generating content that may be harmful, unethical, or illegal.
+ **Data Privacy**: Be cautious when using sensitive or personal data with this model, as it may inadvertently generate sensitive information.


## Troubleshooting : 


+ If you see the following error:
```
Traceback (most recent call last):
File "", line 1, in
File "/transformers/models/auto/auto_factory.py", line 482, in from_pretrained
config, kwargs = AutoConfig.from_pretrained(
File "/transformers/models/auto/configuration_auto.py", line 1022, in from_pretrained
config_class = CONFIG_MAPPING[config_dict["model_type"]]
File "/transformers/models/auto/configuration_auto.py", line 723, in getitem
raise KeyError(key)
KeyError: 'mistral'
```


**Installing transformers from source should solve the issue:**



```
pip install git+https://github.com/huggingface/transformers
```


## Notice : 
**Mistral 7B** is a pretrained base model and therefore does not have any moderation mechanisms.

## License
+ **Model License**: Apache 2.0 License

## Getting Started (with Peft): 

```python
from peft import PeftModel, PeftConfig
from transformers import AutoModelForCausalLM

config = PeftConfig.from_pretrained("ayoubkirouane/Mistral-7b-DZ_Startups")
model = AutoModelForCausalLM.from_pretrained("bn22/Mistral-7B-Instruct-v0.1-sharded")
model = PeftModel.from_pretrained(model, "ayoubkirouane/Mistral-7b-DZ_Startups")
```
{"language": ["en"], "license": "apache-2.0", "library_name": "peft", "datasets": ["TuningAI/Startups_V1"], "pipeline_tag": "text-generation"}
peft text-generation en dataset:TuningAI/Startups_V1 license:apache-2.0 region:us