llama-2 en license:apache-2.0 region:us
---
language:
- en
tags:
- llama-2
license: apache-2.0
---
# Model Card: Curie-7B


## Model Description

Curie-7B is a work-in-progress language model fine-tuned on chemistry & chemical substance data. It will be based on LlaMA-2-7B. This model is being fine-tuned by SynthMind & is meant to only use open-source datasets. This should make the model reproducible by anyone. Nobody is sponsoring the compute (at least for now).

This model will try to obtain long responses & lower hallucination rates. The fine-tuning process will be performed with a TBD sequence length on a 2x 3090 24GB machine (TBC).

## Training Datasets

The Curie-7B model is fine-tuned on the following openly available datasets:

- [WizardLM](https://huggingface.co/datasets/ehartford/WizardLM_alpaca_evol_instruct_70k_unfiltered)
- [GPTeacher-General-Instruct](https://huggingface.co/datasets/teknium/GPTeacher-General-Instruct)
- [metaeval/ScienceQA_text_only](https://huggingface.co/datasets/metaeval/ScienceQA_text_only) - instruct for concise responses
- [openai/summarize_from_feedback](https://huggingface.co/datasets/openai/summarize_from_feedback) - instruct augmented tl;dr summarization
- [camel-ai/math](https://huggingface.co/datasets/camel-ai/math)
- [camel-ai/physics](https://huggingface.co/datasets/camel-ai/physics)
- [camel-ai/chemistry](https://huggingface.co/datasets/camel-ai/chemistry)
- [camel-ai/biology](https://huggingface.co/datasets/camel-ai/biology)
 

## Prompt Format
Chat only style prompts using USER:,ASSISTANT:.

## Benchmarks:
TODO

# Demo
TODO

## Future Plans
As with many other models, it's planned to continue iterating on both more high-quality data, and new data filtering techniques to eliminate lower-quality data going forward. 

## Build / Training procedure
TODO

## Examples
TODO