# 🇧🇷 Opt-6.7B-lora-caramelo 🇧🇷

### Model Description

Opt-6.7B-lora-caramelo is further pre-train [Facebook's OPT-6.78](https://huggingface.co/facebook/opt-6.7b) model using casual language modeling on wikipedia-portuguese version [05/04/2023].

### Limitations and Biases

This model is intended to be used with fine-tuning, supervision, and/or moderation. 
I recommend having a human curate or filter the outputs.


### How to use

```py
import torch
from peft import PeftModel, PeftConfig
from transformers import AutoModelForCausalLM, AutoTokenizer

peft_model_id = "arthurangelici/opt-6.7b-lora-caramelo"
config = PeftConfig.from_pretrained(peft_model_id)
model = AutoModelForCausalLM.from_pretrained(config.base_model_name_or_path, return_dict=True, load_in_8bit=True, device_map='auto')
tokenizer = AutoTokenizer.from_pretrained(config.base_model_name_or_path)

# Load the Lora model
model = PeftModel.from_pretrained(model, peft_model_id)

batch = tokenizer("Caramelo é um simbolo do: ", return_tensors='pt')

with torch.cuda.amp.autocast():
  output_tokens = model.generate(**batch, max_new_tokens=50)

print('\n\n', tokenizer.decode(output_tokens[0], skip_special_tokens=True))
```

### License

The model is licensed under the OPT-6.75B license, Copyright (c) Meta Platforms, Inc. All Rights Reserved.

#### BibTeX entry and citation info
```
@misc{zhang2022opt,
      title={OPT: Open Pre-trained Transformer Language Models}, 
      author={Susan Zhang and Stephen Roller and Naman Goyal and Mikel Artetxe and Moya Chen and Shuohui Chen and Christopher Dewan and Mona Diab and Xian Li and Xi Victoria Lin and Todor Mihaylov and Myle Ott and Sam Shleifer and Kurt Shuster and Daniel Simig and Punit Singh Koura and Anjali Sridhar and Tianlu Wang and Luke Zettlemoyer},
      year={2022},
      eprint={2205.01068},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}
```
{"language": ["pt"], "license": "other", "library_name": "adapter-transformers", "datasets": ["wikipedia"], "pipeline_tag": "text-generation"}
adapter-transformers opt text-generation pt dataset:wikipedia arxiv:2205.01068 license:other region:us
TAGS
#adapter-transformers #opt #text-generation #pt #dataset-wikipedia #arxiv-2205.01068 #license-other #region-us 

# 🇧🇷 Opt-6.7B-lora-caramelo 🇧🇷

### Model Description

Opt-6.7B-lora-caramelo is further pre-train Facebook's OPT-6.78 model using casual language modeling on wikipedia-portuguese version [05/04/2023].

### Limitations and Biases

This model is intended to be used with fine-tuning, supervision, and/or moderation. 
I recommend having a human curate or filter the outputs.


### How to use



### License

The model is licensed under the OPT-6.75B license, Copyright (c) Meta Platforms, Inc. All Rights Reserved.

#### BibTeX entry and citation info

# 🇧🇷 Opt-6.7B-lora-caramelo 🇧🇷 ### Model Description

Opt-6.7B-lora-caramelo is further pre-train Facebook's OPT-6.78 model using casual language modeling on wikipedia-portuguese version [05/04/2023]. ### Limitations and Biases

This model is intended to be used with fine-tuning, supervision, and/or moderation. 
I recommend having a human curate or filter the outputs. ### How to use ### License

The model is licensed under the OPT-6.75B license, Copyright (c) Meta Platforms, Inc. All Rights Reserved. #### BibTeX entry and citation info
TAGS
#adapter-transformers #opt #text-generation #pt #dataset-wikipedia #arxiv-2205.01068 #license-other #region-us 
 # 🇧🇷 Opt-6.7B-lora-caramelo 🇧🇷 ### Model Description

Opt-6.7B-lora-caramelo is further pre-train Facebook's OPT-6.78 model using casual language modeling on wikipedia-portuguese version [05/04/2023]. ### Limitations and Biases

This model is intended to be used with fine-tuning, supervision, and/or moderation. 
I recommend having a human curate or filter the outputs. ### How to use ### License

The model is licensed under the OPT-6.75B license, Copyright (c) Meta Platforms, Inc. All Rights Reserved. #### BibTeX entry and citation info
TAGS
#adapter-transformers #opt #text-generation #pt #dataset-wikipedia #arxiv-2205.01068 #license-other #region-us 
# 🇧🇷 Opt-6.7B-lora-caramelo 🇧🇷### Model Description

Opt-6.7B-lora-caramelo is further pre-train Facebook's OPT-6.78 model using casual language modeling on wikipedia-portuguese version [05/04/2023].### Limitations and Biases

This model is intended to be used with fine-tuning, supervision, and/or moderation. 
I recommend having a human curate or filter the outputs.### How to use### License

The model is licensed under the OPT-6.75B license, Copyright (c) Meta Platforms, Inc. All Rights Reserved.#### BibTeX entry and citation info