
This model is a Portuguese fine-tuned version of the [facebook/opt-125m](https://huggingface.co/facebook/opt-125m). It has undergone additional causal language modeling pre-training with a context size of 512, using an extra 300 million tokens in Portuguese (sampled from mc4). The Wandb report is publicly available at [here](https://api.wandb.ai/links/thiagolaitz1/ths2zi4c). The code for training using Colab pro (A100 - 40GB) can be found [here](https://github.com/thiagolaitz/IA368-search-engines/blob/main/Project%2004/opt_125m_pt_finetuning.ipynb). The total cost for training this model was R$17.40 or $3.37 USD (as of March 2023).

Deterministic use:
```python
from transformers import pipeline

generator = pipeline('text-generation', model="thiagolaitz/opt-125m-pt-finetuned", max_length=30)
generator("Eles brincaram o dia inteiro sob o sol quente, mas")
# Output: Eles brincaram o dia inteiro sob o sol quente, mas não se deixaram levar pelo sol.
```

Top-k sampling:
```python
from transformers import pipeline

generator = pipeline('text-generation', model="thiagolaitz/opt-125m-pt-finetuned", do_sample=True, max_length=30)
generator("Eles brincaram o dia inteiro sob o sol quente, mas")
```
{"language": ["pt"], "library_name": "transformers", "datasets": ["mc4"], "metrics": ["perplexity"]}
transformers pytorch opt text-generation pt dataset:mc4 autotrain_compatible endpoints_compatible text-generation-inference region:us
TAGS
#transformers #pytorch #opt #text-generation #pt #dataset-mc4 #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 


This model is a Portuguese fine-tuned version of the facebook/opt-125m. It has undergone additional causal language modeling pre-training with a context size of 512, using an extra 300 million tokens in Portuguese (sampled from mc4). The Wandb report is publicly available at here. The code for training using Colab pro (A100 - 40GB) can be found here. The total cost for training this model was R$17.40 or $3.37 USD (as of March 2023).

Deterministic use:


Top-k sampling:

TAGS
#transformers #pytorch #opt #text-generation #pt #dataset-mc4 #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 

TAGS
#transformers #pytorch #opt #text-generation #pt #dataset-mc4 #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 
