transformers safetensors mistral text-generation trl conversational en dataset:vpgits/SDGP_Qagen license:mit autotrain_compatible endpoints_compatible text-generation-inference 4-bit awq region:us
---
license: mit
datasets:
- vpgits/SDGP_Qagen
language:
- en
library_name: transformers
pipeline_tag: text-generation
tags:
- trl
---

# Model Card for vpgits/Mistral-7B-v0.1-qagen-v2.1-AWQ

<!-- Provide a quick summary of what the model is/does. -->

vpgits/Mistral-7B-v0.1-qagen-v2.1-AWQ is an experimental model aimed at using LLMs to create MCQ questions in the JSON format by only providing the source text


### Model Description

<!-- Provide a longer summary of what this model is. -->



- **Developed by:** vpgits
- **Language(s) (NLP):** English
- **License:** MIT
- **Finetuned from model :** [mistralai/Mistral-7B-Instruct-v0.1](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1)

### Model Sources [optional]

<!-- Provide the basic links for the model. -->

- **Repository:** [click here](https://github.com/vpgits/sdgp-ml)
- **Demo [optional]:** [click_here](https://quizzifyme.venura.dev/)

## Uses

<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
Generation of MCQ questions as JSON by providing source text


## How to Get Started with the Model

Use the code below to get started with the model.

```
from awq import AutoAWQForCausalLM
from transformers import AutoTokenizer

model = AutoAWQForCausalLM.from_pretrained("vpgits/Mistral-7B-v0.1-qagen-v2.1-AWQ", fuse_layers=True)
tokenizer = AutoTokenizer.from_pretrained("vpgits/Mistral-7B-v0.1-qagen-v2.1-AWQ", trust_remote_code=True)

#preprocess input text
tokens = tokenizer(
    text=eval_prompt, 
    return_tensors='pt'
).input_ids.cuda()


# Generate output
generation_output = model.generate(
    tokens, 
    max_new_tokens=512
)
```