Update README.md

README.md

@@ -1,3 +1,171 @@
----
-license: apache-2.0
----
+---
+license: apache-2.0
+datasets:
+- CohereForAI/aya_dataset
+- argilla/databricks-dolly-15k-curated-multilingual
+- Gael540/dataSet_ens_sup_fr-v1
+- ai2-adapt-dev/flan_v2_converted
+- OpenAssistant/oasst1
+language:
+- fr
+- en
+- de
+- it
+- es
+base_model:
+- OpenLLM-France/Lucie-7B
+pipeline_tag: text-generation
+---
+
+# Model Card for Lucie-7B-Instruct-human-data
+
+* [Model Description](#model-description)
+<!-- * [Uses](#uses) -->
+* [Training Details](#training-details)
+  * [Training Data](#training-data)
+  * [Preprocessing](#preprocessing)
+  * [Training Procedure](#training-procedure)
+<!-- * [Evaluation](#evaluation) -->
+* [Testing the model](#testing-the-model)
+  * [Test in python](#test-in-python)
+  * [Test with ollama](#test-with-ollama)
+  * [Test with vLLM](#test-with-vllm)
+* [Citation](#citation)
+* [Acknowledgements](#acknowledgements)
+* [Contact](#contact)
+
+## Model Description
+
+Lucie-7B-Instruct-human-data is a fine-tuned version of [Lucie-7B](), an open-source, multilingual causal language model created by OpenLLM-France.
+
+Lucie-7B-Instruct-human-data is fine-tuned on human-produced instructions collected either from open annotation campaigns or by applying templates to extant datasets.
+
+
+## Training details
+### Training data
+
+Lucie-7B-Instruct-human-data is trained on the following datasets published by third parties:
+* [Aya Dataset](https://huggingface.co/datasets/CohereForAI/aya_dataset) (English, 3944 samples; French, 1422; German, 241; Italian, 738; Spanish, 3854)
+* [Dolly](https://huggingface.co/datasets/argilla/databricks-dolly-15k-curated-multilingual) (English, French, German, Spanish; 15015 x 4 samples)
+* [ENS](https://huggingface.co/datasets/Gael540/dataSet_ens_sup_fr-v1) (French, 394 samples)
+* [FLAN v2 Converted](https://huggingface.co/datasets/ai2-adapt-dev/flan_v2_converted) (English, 78580 samples)
+* [Open Assistant 1](https://huggingface.co/datasets/OpenAssistant/oasst1) (English, 21151 samples; French, 1223; German, 1515; Italian, 370; Spanish, 14078)
+* [Oracle](https://github.com/opinionscience/InstructionFr/tree/main/wikipedia) (French, 4613 samples)
+* [PIAF](https://www.data.gouv.fr/fr/datasets/piaf-le-dataset-francophone-de-questions-reponses/) (French, 1849 samples)
+
+
+And the following datasets developed for the Lucie instruct models:
+* Croissant Aligned Instruct (French-English, 20K examples sampled randomly from 80K total)
+* Hard-coded prompts concerning OpenLLM and Lucie (based on [allenai/tulu-3-hard-coded-10x](https://huggingface.co/datasets/allenai/tulu-3-hard-coded-10x))
+    * French: openllm_french.jsonl (24x10 samples)
+    * English: openllm_english.jsonl (24x10 samples)
+
+### Preprocessing
+* Filtering by language: Aya Dataset, Dolly and Open Assistant were filtered to keep only English and French samples, respectively.
+* Filtering by keyword: Examples containing assistant responses were filtered out from Open Assistant if the responses contained a keyword from the list [filter_strings](https://github.com/OpenLLM-France/Lucie-Training/blob/98792a1a9015dcf613ff951b1ce6145ca8ecb174/tokenization/data.py#L2012). This filter is designed to remove examples in which the assistant is presented as model other than Lucie (e.g., ChatGPT, Gemma, Llama, ...).
+* Duplicate examples were removed from Open Assistant.
+
+### Training procedure
+
+The model architecture and hyperparameters are the same as for [Lucie-7B](https://huggingface.co/OpenLLM-France/Lucie-7B) during the annealing phase with the following exceptions:
+* context length: 4096 
+* batch size: 1024
+* max learning rate: 3e-5
+* min learning rate: 3e-6
+
+
+## Testing the model
+### Test in python
+
+* [test_transformers_gguf.py](test_transformers_gguf.py): Test GGUF model with `transformers` package (WARNING: loading the model is long)
+
+### Test with ollama
+
+* Download and install [Ollama](https://ollama.com/download)
+* Download the [GGUF model](https://huggingface.co/OpenLLM-France/Lucie-7B-Instruct-v1/resolve/main/Lucie-7B-q4_k_m.gguf)
+* Copy the [`Modelfile`](Modelfile), adpating if necessary the path to the GGUF file (line starting with `FROM`).
+* Run in a shell:
+    * `ollama create -f Modelfile Lucie`
+    * `ollama run Lucie`
+* Once ">>>" appears, type your prompt(s) and press Enter.
+* Optionally, restart a conversation by typing "`/clear`"
+* End the session by typing "`/bye`".
+
+Useful for debug:
+* [How to print input requests and output responses in Ollama server?](https://stackoverflow.com/a/78831840)
+* [Documentation on Modelfile](https://github.com/ollama/ollama/blob/main/docs/modelfile.md#parameter)
+   * Examples: [Ollama model library](https://github.com/ollama/ollama#model-library)
+      * Llama 3 example: https://ollama.com/library/llama3.1
+* Add GUI : https://docs.openwebui.com/
+
+### Test with vLLM
+
+#### 1. Run vLLM Docker Container
+
+Use the following command to deploy the model,
+replacing `INSERT_YOUR_HF_TOKEN` with your Hugging Face Hub token.
+
+```bash
+docker run --runtime nvidia --gpus=all \
+    --env "HUGGING_FACE_HUB_TOKEN=INSERT_YOUR_HF_TOKEN" \
+    -p 8000:8000 \
+    --ipc=host \
+    vllm/vllm-openai:latest \
+    --model OpenLLM-France/Lucie-7B-Instruct-v1
+```
+
+#### 2. Test using OpenAI Client in Python
+
+To test the deployed model, use the OpenAI Python client as follows:
+
+```python
+from openai import OpenAI
+
+# Initialize the client
+client = OpenAI(base_url='http://localhost:8000/v1', api_key='empty')
+
+# Define the input content
+content = "Hello Lucie"
+
+# Generate a response
+chat_response = client.chat.completions.create(
+    model="OpenLLM-France/Lucie-7B-Instruct-v1",
+    messages=[
+        {"role": "user", "content": content}
+    ],
+)
+print(chat_response.choices[0].message.content)
+```
+
+## Citation
+
+TODO
+
+## Acknowledgements
+
+This work was performed using HPC resources from GENCI–IDRIS (Grant 2024-GC011015444).
+
+Lucie-7B was created by members of [LINAGORA](https://labs.linagora.com/) and the [OpenLLM-France](https://www.openllm-france.fr/) community, including in alphabetical order:
+Olivier Gouvert (LINAGORA),
+Ismaïl Harrando (LINAGORA/SciencesPo), 
+Julie Hunter (LINAGORA),
+Jean-Pierre Lorré (LINAGORA),
+Jérôme Louradour (LINAGORA),
+Michel-Marie Maudet (LINAGORA), and
+Laura Rivière (LINAGORA).
+
+
+We thank 
+Clément Bénesse (Opsci), 
+Christophe Cerisara (LORIA),
+Evan Dufraisse (CEA),
+Guokan Shang (MBZUAI), 
+Joël Gombin (Opsci), 
+Jordan Ricker (Opsci), 
+and
+Olivier Ferret (CEA) 
+for their helpful input.
+
+## Contact
+
+[email protected]