Update README.md

README.md

@@ -1,21 +1,47 @@
 ---
 library_name: peft
+tags:
+- tiiuae-falcon-180B
+- code
+- instruct
+- databricks-dolly-15k
+- falcon-180B
+datasets:
+- databricks/databricks-dolly-15k
+base_model: tiiuae/falcon-180B
 ---
-## Training procedure
 
+For our finetuning process, we used the tiiuae/falcon-180B model and the Databricks-dolly-15k dataset. This dataset is a rich corpus of over 15,000 records, painstakingly created by the collaborative efforts of thousands of Databricks employees. The goal was to enable large language models to emulate the magical interactivity of ChatGPT.
 
-The following `bitsandbytes` quantization config was used during training:
-- quant_method: bitsandbytes
-- load_in_8bit: False
-- load_in_4bit: True
-- llm_int8_threshold: 6.0
-- llm_int8_skip_modules: None
-- llm_int8_enable_fp32_cpu_offload: False
-- llm_int8_has_fp16_weight: False
-- bnb_4bit_quant_type: nf4
-- bnb_4bit_use_double_quant: True
-- bnb_4bit_compute_dtype: bfloat16
-### Framework versions
+The contributors were asked to create prompt / response pairs spread across eight different instruction categories. This included the seven categories outlined in the InstructGPT paper, as well as an open-ended, free-form category. To ensure the uniqueness and authenticity of the data, contributors were instructed to abstain from using information from any online source, with the sole exception being Wikipedia (for specific subsets of instruction categories). They were also explicitly instructed to avoid using generative AI in formulating instructions or responses.
 
+During the data generation process, contributors had the opportunity to answer questions posed by other contributors. They were prompted to rephrase the original question and encouraged to select only those questions they were confident they could answer correctly.
 
-- PEFT 0.5.0
+In certain categories, contributors were asked to provide reference texts sourced from Wikipedia. These references (indicated by the context field in the dataset) may contain bracketed Wikipedia citation numbers (e.g. [42]). We recommend users to remove these for downstream applications.
+
+This finetuning process was carried out using [MonsterAPI](https://monsterapi.ai)'s no-code [LLM finetuner](https://docs.monsterapi.ai/fine-tune-a-large-language-model-llm). The session lasted for 41.7 hours and costed us `$184.314`, running on 2x A100 80GB GPUs.
+
+#### Hyperparameters & Run details:
+- Model Path: tiiuae/falcon-180B
+- Dataset: databricks/databricks-dolly-15k
+- Learning rate: 0.0002
+- Number of epochs: 1
+- Data split: Training: 90% / Validation: 10%
+- Gradient accumulation steps: 1
+
+license: apache-2.0
+---
+
+######
+
+Prompt Used: 
+
+```
+### INSTRUCTION:
+[instruction]
+
+[context]
+
+### RESPONSE:
+[response]
+```