
# GPT2-borges (gpt2-borges)


## Overview

- **Architecture:** gpt2-base
- **Language:** Spanish
- **Task:** text-generation
- **Data:** Borges Works

## Model description

**GPT2-borges** is a transformer-based model for the Spanish language. It is based on the PlanTL-GOB-ES/gpt2-base-bne model and has been pre-trained using a curated dataset consisting of the complete works of Jorge Luis Borges, a renowned Argentine writer.

## Intended uses and limitations
You can use the raw model for text generation or fine-tune it to a downstream task.

## How to Use
Here is how to use this model:

You can use this model directly with a pipeline for text generation. Since the generation relies on some randomness, we set a seed for reproducibility:

```python
from transformers import AutoTokenizer, AutoModelForCausalLM
model_name = 'lucasbiagettia/gpt2-base-borges'
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)
```

```python
input_text = "La arena me recuerda el profundo dolor de la nostalgia"

input_ids = self.tokenizer.encode(input_text, return_tensors="pt")
attention_mask = torch.ones(input_ids.shape, dtype=torch.long)
    

generated_text = self.model.generate(
            input_ids=input_ids,
            attention_mask=attention_mask,
            max_new_tokens = 100,
            num_return_sequences=1,
            no_repeat_ngram_size=6,
            top_k=35,
            top_p=0.95,
            temperature=0.8,
            pad_token_id=50256,
            do_sample=True,
)
```

## Training
was trained with the following dataset:

https://github.com/lucasbiagettia/borges_plain_text_dataset
{"language": ["es"], "license": "apache-2.0", "tags": ["borges", "spanish", "text-generation"], "datasets": ["borges_works"], "widget": [{"text": "El modelo del lenguaje GPT es capaz de"}, {"text": "Las obras de Borges son una fuente rica de conocimiento y creatividad"}]}
transformers safetensors gpt2 text-generation borges spanish es dataset:borges_works license:apache-2.0 autotrain_compatible endpoints_compatible text-generation-inference region:us