
## Mistral-Syndicate-7B


<img src="https://cdn-uploads.huggingface.co/production/uploads/64712903631bb7660aaa7183/yMzb5tbgygtlHnoyR37YO.png" width="600px" />

## Model Description:

Mistral Syndicate is in no way a state-of-the-art model, rather it is a fine-tuning experiment to explore
the training dynamics specific to large language models. The dataset used in finetuning was generated via
a "syndicate" of other open language models both of similar parameter size and larger. Each model would generate a
response for a given instruction, and the group would vote on which model's response was best.

The instruction inputs used for the output label synthesis were a curated subset of [VMWare/open-instruct](https://huggingface.co/datasets/VMware/open-instruct)
with additional instructions synthesized from scratch.

## Prompt template

With context
```
Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.

### Instruction:

### Input:

### Response:
```

Without context
```
Below is an instruction that describes a task. Write a response that appropriately completes the request.

### Instruction:

### Response:
```

## Evaluation Results
_12.30.23_
| Benchmark  | Result |
|------------|--------|
| ARC        | 60.84  |
| HellaSwag  | 82.91  |
| MMLU       | 60.83  |
| TruthfulQA | 43.71  |
| Winogrande | 78.61  |
| GSM8K      | 44.50  |


{"language": ["en"], "license": "apache-2.0", "tags": ["mistral-7b", "instruct", "finetune", "synthetic data", "distillation"], "base_model": "mistralai/Mistral-7B-v0.1", "model-index": [{"name": "Mistral-Syndicate-7B", "results": []}]}
transformers pytorch safetensors mistral text-generation mistral-7b instruct finetune synthetic data distillation en base_model:mistralai/Mistral-7B-v0.1 license:apache-2.0 autotrain_compatible endpoints_compatible text-generation-inference region:us