

# OPT-Alpaca-125M

This model is a fine-tuned version of [facebook/opt-125m](https://huggingface.co/facebook/opt-125m) on the [tatsu-lab/alpaca](https://huggingface.co/datasets/tatsu-lab/alpaca) dataset. This is a test to check if the concept of Databrick's [databricks/dolly-v1-6b](https://huggingface.co/databricks/dolly-v1-6b) can be applied to smaller models.

## Model description

This is an LLM trained to follow instructions, similar to ChatGPT, except much much smaller.

## Sample Use

```python
from transformers import pipeline

generator = pipeline('text-generation', model='linkanjarad/OPT-Alpaca-125M')

prompt = '''### Instruction:
What is the capital of Australia?

### Response:'''

output = generator(prompt, do_sample=True, min_length=10, max_length=50, temperature=0.5)
print(output[0]['generated_text'])
```

## Intended uses & limitations

The model works best when given input in the format of the dataset it was trained on. Here is an example of the format of the [tatsu-lab/alpaca](https://huggingface.co/datasets/tatsu-lab/alpaca) dataset:

```
### Instruction:
What are the three primary colors?

### Response:
The three primary colors are red, blue, and yellow.
```

Here are a few sample model outputs (`temperature=0.7`):

```
### Instruction:
What is the capital of Australia?

### Response:
The capital of Australia is Canberra.
```

```
### Instruction:
How do rainbows form?

### Response:
Rainbows form when the light of a sun enters the earth's atmosphere. The sun's rays are absorbed by the atmosphere, creating rainbows. The sun's light is then used to create a rainbow, which is a result of the Earth's rotation.
```

```
### Instruction:
Write me a poem about pancakes in the style of Dr. Seuss.

### Response:
Pancakes are like a wonderful miracle. They give us a beautiful and delicious breakfast.
Pancakes go from day to day,
But they also bring us pleasure.

I'm so glad I made the time to make my own
Pancakes are a delight for me and my family.
I'm going to make a special treat for you
```

```
### Instruction:
If you put an egg on the tip of a pencil, what will happen?

### Response:
An egg will start to move around the pencil without a trace.
```

```
### Instruction:
Write me code in Python that prints out the first ten numbers in the Fibonacci sequence.

### Response:
def number_primes(fibonacci):
    for i in range(10):
       for j in range(10):
          print(i)
    return i
```

```
### Instruction:
Write an article about card games with SEO.

### Response:
Card games are a popular game that allow players to play the game of cards with friends. The game is a puzzle game with players finding the correct cards that will be used to win a game. Players can use a variety of strategies to reach the right goal and win the game. Card games provide a lot of fun, and are a great way of providing a fun and engaging experience.
```

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 2
- eval_batch_size: 2
- seed: 42
- gradient_accumulation_steps: 64
- total_train_batch_size: 128
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.15
- num_epochs: 3

### Framework versions

- Transformers 4.27.4
- Pytorch 1.13.0
- Datasets 2.1.0
- Tokenizers 0.13.2
{"license": "other", "tags": ["generated_from_trainer"], "datasets": ["tatsu-lab/alpaca"], "widget": [{"text": "### Instruction:\nWrite a poem about pancakes.\n\n### Response:"}], "model-index": [{"name": "OPT-Alpaca-125M", "results": []}]}
transformers pytorch opt text-generation generated_from_trainer dataset:tatsu-lab/alpaca license:other autotrain_compatible endpoints_compatible text-generation-inference region:us
TAGS
#transformers #pytorch #opt #text-generation #generated_from_trainer #dataset-tatsu-lab/alpaca #license-other #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 



# OPT-Alpaca-125M

This model is a fine-tuned version of facebook/opt-125m on the tatsu-lab/alpaca dataset. This is a test to check if the concept of Databrick's databricks/dolly-v1-6b can be applied to smaller models.

## Model description

This is an LLM trained to follow instructions, similar to ChatGPT, except much much smaller.

## Sample Use



## Intended uses & limitations

The model works best when given input in the format of the dataset it was trained on. Here is an example of the format of the tatsu-lab/alpaca dataset:



Here are a few sample model outputs ('temperature=0.7'):













### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 2
- eval_batch_size: 2
- seed: 42
- gradient_accumulation_steps: 64
- total_train_batch_size: 128
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.15
- num_epochs: 3

### Framework versions

- Transformers 4.27.4
- Pytorch 1.13.0
- Datasets 2.1.0
- Tokenizers 0.13.2
# OPT-Alpaca-125M

This model is a fine-tuned version of facebook/opt-125m on the tatsu-lab/alpaca dataset. This is a test to check if the concept of Databrick's databricks/dolly-v1-6b can be applied to smaller models. ## Model description

This is an LLM trained to follow instructions, similar to ChatGPT, except much much smaller. ## Sample Use ## Intended uses & limitations

The model works best when given input in the format of the dataset it was trained on. Here is an example of the format of the tatsu-lab/alpaca dataset:



Here are a few sample model outputs ('temperature=0.7'): ### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 2
- eval_batch_size: 2
- seed: 42
- gradient_accumulation_steps: 64
- total_train_batch_size: 128
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.15
- num_epochs: 3 ### Framework versions

- Transformers 4.27.4
- Pytorch 1.13.0
- Datasets 2.1.0
- Tokenizers 0.13.2
TAGS
#transformers #pytorch #opt #text-generation #generated_from_trainer #dataset-tatsu-lab/alpaca #license-other #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 
 # OPT-Alpaca-125M

This model is a fine-tuned version of facebook/opt-125m on the tatsu-lab/alpaca dataset. This is a test to check if the concept of Databrick's databricks/dolly-v1-6b can be applied to smaller models. ## Model description

This is an LLM trained to follow instructions, similar to ChatGPT, except much much smaller. ## Sample Use ## Intended uses & limitations

The model works best when given input in the format of the dataset it was trained on. Here is an example of the format of the tatsu-lab/alpaca dataset:



Here are a few sample model outputs ('temperature=0.7'): ### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 2
- eval_batch_size: 2
- seed: 42
- gradient_accumulation_steps: 64
- total_train_batch_size: 128
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.15
- num_epochs: 3 ### Framework versions

- Transformers 4.27.4
- Pytorch 1.13.0
- Datasets 2.1.0
- Tokenizers 0.13.2
TAGS
#transformers #pytorch #opt #text-generation #generated_from_trainer #dataset-tatsu-lab/alpaca #license-other #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 
# OPT-Alpaca-125M

This model is a fine-tuned version of facebook/opt-125m on the tatsu-lab/alpaca dataset. This is a test to check if the concept of Databrick's databricks/dolly-v1-6b can be applied to smaller models.## Model description

This is an LLM trained to follow instructions, similar to ChatGPT, except much much smaller.## Sample Use## Intended uses & limitations

The model works best when given input in the format of the dataset it was trained on. Here is an example of the format of the tatsu-lab/alpaca dataset:



Here are a few sample model outputs ('temperature=0.7'):### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 2
- eval_batch_size: 2
- seed: 42
- gradient_accumulation_steps: 64
- total_train_batch_size: 128
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.15
- num_epochs: 3### Framework versions

- Transformers 4.27.4
- Pytorch 1.13.0
- Datasets 2.1.0
- Tokenizers 0.13.2