
This is a strong TinyLlama-1.1B-Chat-Model trained using a modified version of PPO + DPO algo



*****Score rated by ChatGPT + Claude + Gemini (max-score is 3)*****


| Characteristic  | Baseline (TinyLlama/TinyLlama-1.1B-Chat-v1.0)    | Ours (TinyLlama-1.1B-ChatStrong-DPO-PPO) |
|-----------------|--------------------------------------------------|--------------------------------------------------------|
| Fluency         | 2                                                | 3                                                      |
| Coherence       | 1                                                | 3                                                      |
| Non-repetition  | 1                                                | 3                                                      |
| Engagement      | 1                                                | 2                                                      |
| Creativity      | 1                                                | 2                                                      |
| Accuracy        | 2                                                | 2                                                      |
| Relevance       | 2                                                | 3                                                      |
| Knowledge       | 2                                                | 2                                                      |
| Common sense    | 1                                                | 2                                                      |
| Empathy         | 1                                                | 2                                                      |
| Humor           | 1                                                | 2                                                      |
| Logic           | 1                                                | 2                                                      |
| Grammar         | 2                                                | 3                                                      |

*Note: score is based on prompt generated by GPT4. 

```python
# Install transformers from source - only needed for versions <= v4.34
# pip install git+https://github.com/huggingface/transformers.git
# pip install accelerate

import torch
from transformers import pipeline

pipe = pipeline("text-generation", model="shitshow123/TinyLlama-1.1B-ChatStrong-DPO-PPO", torch_dtype=torch.bfloat16, device_map="auto")

# We use the tokenizer's chat template to format each message - see https://huggingface.co/docs/transformers/main/en/chat_templating
messages = [
    {
        "role": "system",
        "content": "You are a friendly chatbot who always responds in the style of a pirate",
    },
    {"role": "user", "content": "How many helicopters can a human eat in one sitting?"},
]
prompt = pipe.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
outputs = pipe(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
#print(outputs[0]["generated_text"])
output_text = outputs[0]["generated_text"]
words = output_text.split()

# Insert a newline character every 50 words
formatted_output = ""
for i in range(0, len(words), 10):
    formatted_output += ' '.join(words[i:i+10]) + "\n"

print(formatted_output)
# ...
```

## Example Output by this model:
<|system|> You are a friendly chatbot who always responds in
the style of a pirate</s> ******<|user|> Provide a summary of
the latest developments in renewable energy technologies, focusing on solar
and wind power advancements in the past year.</s> <|assistant|>****** In
the past year, significant advancements have been made in the
renewable energy sector, particularly in the areas of solar and
wind power. Here's a summary of some of the notable
developments: 1. Solar: - Solar panels have become more efficient,
with a record-breaking 29.2% conversion efficiency achieved in 2020. -
The world's largest solar farm, located in China, has a
capacity of 1.2 GW, making it the largest solar power
plant in the world. - The European Union has set
a target of achieving 32% renewable energy by 2030, with
solar power expected to play a significant role in achieving
this goal. 2. Wind: - Wind turbines have become more
efficient, with a record-breaking 1.2 GW of wind power capacity
installed in 2020. - The world's largest offshore wind farm,
located in the UK, has a capacity of 670 MW.
- The US has set a target of achieving 100
GW of offshore wind power by 2030, with the first
offshore wind farm in the country, located in Massachusetts, expected
to be operational by 2023. These advancements in renewable energy
technologies have the potential to significantly reduce greenhouse gas emissions
and improve energy security in the coming years.


## Example Output by the baseline TinyLamma model (see the repetition of Novemeber 2020):
<|system|> You are a friendly chatbot who always responds in
the style of a pirate</s> ******<|user|> Provide a summary of
the latest developments in renewable energy technologies, focusing on solar
and wind power advancements in the past year.</s> <|assistant|>****** In
the past year, there have been significant advancements in renewable
energy technologies, particularly in solar and wind power. Here's a
summary: 1. Solar: - Solar panel efficiency has increased by
10% in the past year, according to a report by
the International Renewable Energy Agency (IRENA). - The world's largest
solar power plant, the 1.2 GW Solar Park in Abu
Dhabi, opened in November 2020. - The world's largest solar
power plant, the 1.2 GW Solar Park in Abu Dhabi,
opened in November 2020. It is expected to generate enough
electricity to power 500,000 homes. - The world's largest solar
power plant, the 1.2 GW Solar Park in Abu Dhabi,
opened in November 2020. It is expected to generate enough
electricity to power 500,000 homes. - The world's largest solar
power plant, the 1.2 GW Solar Park in Abu Dhabi,
opened in November 2020. It is expected to generate enough
electricity to power 500,000 homes. 2. Wind: - The world's
largest wind farm, the Hornsdale Power Reserve in South Australia,
opened in November 2020. It is expected to generate enough
electricity to power 100,000 homes. - The world's largest wind
farm, the Hornsdale Power Reserve in South Australia, opened in
November 2020. It is expected to generate enough electricity to
power 100,000 homes. - The world's largest wind farm, the
Hornsdale Power Reserve in South Australia, opened in November 2020.
It is expected to generate enough electricity to power 100,000
homes. These advancements in solar and wind power technologies have
the potential to significantly reduce greenhouse gas emissions and improve
energy security.

{"license": "apache-2.0"}
transformers safetensors llama feature-extraction license:apache-2.0 endpoints_compatible text-generation-inference region:us