
# Model Card for Model ID

<!-- Provide a quick summary of what the model is/does. -->
This Model is Finetuned for Document Question and Answering purpose Trained on the yahma/alpaca-cleaned(https://huggingface.co/TheBloke/zephyr-7B-beta-GPTQ) dataset.


## Model Details

### Training hyperparameters

The following hyperparameters were used during training:

-gradient_accumulation_steps=1,

-warmup_steps=5,

-max_steps=20,

-learning_rate=2e-4,

-fp16=not torch.cuda.is_bf16_supported(),

-bf16=torch.cuda.is_bf16_supported(),

-logging_steps=1,

-optim="adamw_8bit",

-weight_decay=0.01,

-lr_scheduler_type="linear",

-seed=3407,

- ### Framework versions

- PEFT 0.7.1
- Transformers 4.36.0
- Pytorch 2.0.0
- Datasets 2.16.1
- Tokenizers 0.15.0


{"language": ["en"], "license": "mit", "library_name": "transformers", "tags": ["PEFT", "mistral", "sft", "TensorBoard ", "Safetensors", " trl", "generated_from_trainer 4-bit", " precision"], "datasets": ["yahma/alpaca-cleaned"], "pipeline_tag": "question-answering"}
transformers safetensors mistral text-generation PEFT sft TensorBoard  Safetensors  trl generated_from_trainer 4-bit  precision question-answering en dataset:yahma/alpaca-cleaned license:mit autotrain_compatible endpoints_compatible text-generation-inference 4-bit region:us