
The BERT **large**-**cased** model trained on all Romanian **tweets from 2008 to 2022**.


# How to use

```python
import torch
from transformers import AutoTokenizer, AutoModel

# Load the tokenizer and the model
tokenizer = AutoTokenizer.from_pretrained("Iulian277/ro-bert-tweet-large")
model = AutoModel.from_pretrained("Iulian277/ro-bert-tweet-large")

# Sanitize the input
!pip install emoji
from normalize import normalize # Use the `normalize` function from the `normalize.py` script
normalized_text = normalize("Salut, ce faci?")

# Tokenize the sentence and run through the model
input_ids = torch.tensor(tokenizer.encode(normalized_text, add_special_tokens=True)).unsqueeze(0) # Batch size 1
outputs = model(input_ids)

# Get encoding
last_hidden_states = outputs[0] # The last hidden-state is the first element of the output tuple
```

**Always** use the `normalize.py` script included in the repository to sanitize you input text, before feeding the tokenizer.
Otherwise you will decrease the performance due to the [UNK] tokens.


# Acknowledgements

We'd like to thank [TPU Research Cloud](https://sites.research.google/trc/about/) for helping us out with the TPU compute power needed to pretrain RoBERTweet models.

{"language": "ro", "license": "mit", "tags": ["bert", "fill-mask"], "mask_token": "[MASK]"}
transformers pytorch bert fill-mask ro license:mit endpoints_compatible region:us
TAGS
#transformers #pytorch #bert #fill-mask #ro #license-mit #endpoints_compatible #region-us 


The BERT large-cased model trained on all Romanian tweets from 2008 to 2022.


# How to use



Always use the 'URL' script included in the repository to sanitize you input text, before feeding the tokenizer.
Otherwise you will decrease the performance due to the [UNK] tokens.


# Acknowledgements

We'd like to thank TPU Research Cloud for helping us out with the TPU compute power needed to pretrain RoBERTweet models.

# How to use



Always use the 'URL' script included in the repository to sanitize you input text, before feeding the tokenizer.
Otherwise you will decrease the performance due to the [UNK] tokens. # Acknowledgements

We'd like to thank TPU Research Cloud for helping us out with the TPU compute power needed to pretrain RoBERTweet models.
TAGS
#transformers #pytorch #bert #fill-mask #ro #license-mit #endpoints_compatible #region-us 
 # How to use



Always use the 'URL' script included in the repository to sanitize you input text, before feeding the tokenizer.
Otherwise you will decrease the performance due to the [UNK] tokens. # Acknowledgements

We'd like to thank TPU Research Cloud for helping us out with the TPU compute power needed to pretrain RoBERTweet models.
TAGS
#transformers #pytorch #bert #fill-mask #ro #license-mit #endpoints_compatible #region-us 
# How to use



Always use the 'URL' script included in the repository to sanitize you input text, before feeding the tokenizer.
Otherwise you will decrease the performance due to the [UNK] tokens.# Acknowledgements

We'd like to thank TPU Research Cloud for helping us out with the TPU compute power needed to pretrain RoBERTweet models.