
# CatastroBERT a model for Extreme weather events detection in french text

This model aims to facilitate the detection of paragraphs or articles relevant to extreme weather events
in french text. It is based on the [camembert-base](https://huggingface.co/camembert-base) model and was trained on manually anotated data (articles summaries) from the Gazette de Lausanne archives  collected by [impresso](https://impresso-project.ch/)

<div align=center>
    <img src="images/bert_illustration.png" width="500" height="500" />
</div>

## Model Description

- **Developed by:** Lucas Nicolas
- **Language(s) (NLP):** French
- **Finetuned from model :** [camembert-base](https://huggingface.co/camembert-base) (RoBERTa Checkpoint)

- **Repository:** Check the [CatastroBERT](https://github.com/dh-epfl-students/dhlab-CatastroBERT) github page for more usage examples and information.

## Usage

### In Transformers

```python
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch


model_name = "epfl-dhlab/CatastroBERT"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification(model_name)

def predict(text):
    # Prepare the text data
    inputs = tokenizer.encode_plus(
        text,
        None,
        add_special_tokens=True,
        return_token_type_ids=True,
        padding=True,
        max_length=512,
        truncation=True,
        return_tensors='pt'
    )

    ids = inputs['input_ids'].to('cuda' if torch.cuda.is_available() else 'cpu')
    mask = inputs['attention_mask'].to('cuda' if torch.cuda.is_available() else 'cpu')

    # Get predictions
    with torch.no_grad():
        outputs = model(ids, mask)
        logits = outputs.logits

    # Apply sigmoid function to get probabilities
    probs = torch.sigmoid(logits).cpu().numpy()

    # Return the probability of the class (1)
    return probs[0][0]

#example usage 
text = "Un violent ouragan du sud-ouest est passé cette nuit sur Lausanne."
print(f"Prediction: {predict(text)}")
```

### Training Data

This model was trained on manually a manually anotated dataset (articles summaries) curated from the Gazette de Lausanne archives  collected by the [impresso](https://impresso-project.ch/) project. The dataset is composed of 4500 articles summaries of which 3500 were used for training and 1000 for validation.

## Environmental Impact

Carbon emissions estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).

- **Hardware Type:** RTX 3090
- **Hours used:** 26
- **Carbon Emitted:** 0.07 kg CO2
{"language": ["fr"], "license": "apache-2.0", "library_name": "transformers", "tags": ["climate"]}
transformers safetensors camembert text-classification climate fr arxiv:1910.09700 license:apache-2.0 autotrain_compatible endpoints_compatible region:us