Update README.md

README.md

@@ -1,22 +1,232 @@
+
 ---
-base_model: unsloth/phi-3-mini-4k-instruct-bnb-4bit
-language:
-- en
-license: apache-2.0
-tags:
-- text-generation-inference
-- transformers
-- unsloth
-- mistral
-- trl
+library_name: transformers
+tags: [Text Generation, Question-Answering]
+inference: false
 ---
 
+
 # Uploaded  model
 
 - **Developed by:** YuvrajSingh9886
 - **License:** apache-2.0
 - **Finetuned from model :** unsloth/phi-3-mini-4k-instruct-bnb-4bit
 
+
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+It's a fine-tuned version of Phi-2 model by Microsoft on [Alpaca-Cleaned-52k](yahma/alpaca-cleaned).
+
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+The above model, with applicable changes to the generation_config file, passed to model.generate() function can lead to the generation of better results which could then be used for Health Counseling Chatbot dev.
+
+
+
+## Bias, Risks, and Limitations
+
+The model was developed as a proof-of-concept type hobby project and is not intended to be used without careful consideration of its implications.
+
+[More Information Needed]
+
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+### Load in the model using the BitsandBytes library
+
+```python
+pip install bitsandbytes
+```
+
+#### Load model from Hugging Face Hub with model name and bitsandbytes configuration
+
+```python
+
+def load_model_tokenizer(model_name: str, bnb_config: BitsAndBytesConfig) -> Tuple[AutoModelForCausalLM, AutoTokenizer]:
+    """
+    Load the model and tokenizer from the HuggingFace model hub using quantization.
+
+    Args:
+        model_name (str): The name of the model.
+        bnb_config (BitsAndBytesConfig): The quantization configuration of BitsAndBytes.
+
+    Returns:
+        Tuple[AutoModelForCausalLM, AutoTokenizer]: The model and tokenizer.
+    """
+
+
+    model = AutoModelForCausalLM.from_pretrained(
+        model_name,
+        quantization_config = bnb_config,
+        # device_map = "auto",
+        torch_dtype="auto",
+        trust_remote_code=True
+    )
+
+    tokenizer = AutoTokenizer.from_pretrained(model_name, use_auth_token = True, trust_remote_code=True)
+
+    tokenizer.pad_token = tokenizer.eos_token
+
+    return model, tokenizer
+
+
+bnb_config = BitsAndBytesConfig(
+        load_in_4bit = load_in_4bit,
+        bnb_4bit_use_double_quant = bnb_4bit_use_double_quant,
+        bnb_4bit_quant_type = bnb_4bit_quant_type,
+        bnb_4bit_compute_dtype = bnb_4bit_compute_dtype,
+    )
+
+model, tokenizer = load_model_tokenizer(model_name, bnb_config)
+
+```
+
+```python
+
+new_model = "YuvrajSingh9886/medicinal-QnA-phi2-custom"
+
+prompt = "I have been feeling more and more down for over a month. I have started having trouble sleeping due to panic attacks, but they are almost never triggered by something that I know of."
+
+tokens = tokenizer(f"### Question: {prompt}", return_tensors='pt').to('cuda')
+tokenizer.pad_token = tokenizer.eos_token
+outputs = model.generate(**tokens, max_new_tokens=1024, num_beams=5,
+    no_repeat_ngram_size=2,
+    early_stopping=True
+                         )
+print(tokenizer.batch_decode(outputs,skip_special_tokens=True)[0])
+
+```
+
+## Training Details
+
+### Training Data
+
+
+#### Hardware
+
+Epcohs: 10
+Hardware: (1) RTX 4090 (24GB VRAM) 48GB 8vCPU (RAM)
+          Hard Disk: 40GB
+
+
+[More Information Needed]
+
+### Training Procedure
+
+QLoRA was used for quantization purposes.
+
+Phi-2 model from Huggingface with BitsandBytes support
+
+
+#### Preprocessing [optional]
+
+```python
+
+def format_phi2(row):
+    question = row['Context']
+    answer = row['Response']
+
+#     text = f"[INST] {question} [/INST] {answer}".replace('\xa0', ' ')
+    text = f"### Question: {question}\n ### Answer: {answer}"
+
+    return text
+```
+
+#### Training Hyperparameters
+
+
+LoRA config-
+```bash
+# LoRA attention dimension (int)
+lora_r = 64
+
+# Alpha parameter for LoRA scaling (int)
+lora_alpha = 16
+
+# Dropout probability for LoRA layers (float)
+lora_dropout = 0.05
+
+# Bias (string)
+bias = "none"
+
+# Task type (string)
+task_type = "CAUSAL_LM"
+
+# Random seed (int)
+seed = 33
+```
+
+Phi-2 config-
+
+```bash
+# Batch size per GPU for training (int)
+per_device_train_batch_size = 6
+
+# Number of update steps to accumulate the gradients for (int)
+gradient_accumulation_steps = 2
+
+# Initial learning rate (AdamW optimizer) (float)
+learning_rate = 2e-4
+
+# Optimizer to use (string)
+optim = "paged_adamw_8bit"
+
+# Number of training epochs (int)
+num_train_epochs = 4
+
+# Linear warmup steps from 0 to learning_rate (int)
+warmup_steps = 10
+
+# Enable fp16/bf16 training (set bf16 to True with an A100) (bool)
+fp16 = True
+
+# Log every X updates steps (int)
+logging_steps = 100
+
+#L2 regularization(prevents overfitting)
+weight_decay=0.0
+
+#Checkpoint saves
+save_strategy="epoch"
+```
+
+BnB config
+
+```bash
+# Activate 4-bit precision base model loading (bool)
+load_in_4bit = True
+
+# Activate nested quantization for 4-bit base models (double quantization) (bool)
+bnb_4bit_use_double_quant = True
+
+# Quantization type (fp4 or nf4) (string)
+bnb_4bit_quant_type = "nf4"
+
+# Compute data type for 4-bit base models
+bnb_4bit_compute_dtype = torch.bfloat16
+
+```
+
+### Results
+
+Training loss: 2.229
+Validation loss: 2.223
+
+
+## More Information [optional]
+
+[Phi-2](https://huggingface.co/microsoft/phi-2)
+
+## Model Card Authors [optional]
+
+[YuvrajSingh9886](https://huggingface.co/YuvrajSingh9886)
+
+
 This mistral model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
 
 [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)